Siamese Neural Networks for EEG-based Brain-computer Interfaces

Motivated by the inconceivable capability of human brain in simultaneously processing multi-modal signals and its real-time feedback to the outer world events, there has been a surge of interest in establishing a communication bridge between the human brain and a computer, which are referred to as Brain-computer Interfaces (BCI). To this aim, monitoring the electrical activity of brain through Electroencephalogram (EEG) has emerged as the prime choice for BCI systems. To discover the underlying and specific features of brain signals for different mental tasks, a considerable number of research works are developed based on statistical and data-driven techniques. However, a major bottleneck in development of practical and commercial BCI systems is their limited performance when the number of mental tasks for classification is increased. In this work, we propose a new EEG processing and feature extraction paradigm based on Siamese neural networks, which can be conveniently merged and scaled up for multi-class problems. The idea of Siamese networks is to train a double-input neural network based on a contrastive loss-function, which provides the capability of verifying if two input EEG trials are from the same class or not. In this work, a Siamese architecture, which is developed based on Convolutional Neural Networks (CNN) and provides a binary output on the similarity of two inputs, is combined with One vs. Rest (OVR) and One vs. One (OVO) techniques to scale up for multi-class problems. The efficacy of this architecture is evaluated on a 4-class Motor Imagery (MI) dataset from BCI Competition IV2a and the results suggest a promising performance compared to its counterparts.

[1]  J. Wolpaw,et al.  Brain–computer interfaces in neurological rehabilitation , 2008, The Lancet Neurology.

[2]  A. Vučković,et al.  Is Implicit Motor Imagery a Reliable Strategy for a Brain–Computer Interface? , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Konstantinos N. Plataniotis,et al.  Separable Common Spatio-Spectral Patterns for Motor Imagery BCI Systems , 2016, IEEE Transactions on Biomedical Engineering.

[4]  Marta Z. Kwiatkowska,et al.  Calibrating the Classifier: Siamese Neural Network Architecture for End-to-End Arousal Recognition from ECG , 2018, LOD.

[5]  Deniz Erdogmus,et al.  The Future of Human-in-the-Loop Cyber-Physical Systems , 2013, Computer.

[6]  Fabio Babiloni,et al.  Passive BCI in Operational Environments: Insights, Recent Advances, and Future Trends , 2017, IEEE Transactions on Biomedical Engineering.

[7]  Carole Lartizien,et al.  Regularized siamese neural network for unsupervised outlier detection on brain multiparametric magnetic resonance imaging: Application to epilepsy lesion screening , 2019, Medical Image Anal..

[8]  T. Ward,et al.  Brain computer interfaces for neurorehabilitation – its current status as a rehabilitation strategy post-stroke. , 2015, Annals of physical and rehabilitation medicine.

[9]  Heung-Il Suk,et al.  A Novel Bayesian Framework for Discriminative Feature Extraction in Brain-Computer Interfaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[12]  Ahmed Fares,et al.  EEG-based image classification via a region-level stacked bi-directional deep learning framework , 2019, BMC Medical Informatics Decis. Mak..

[13]  José Luis Pons Rovira,et al.  A Closed-Loop Brain–Computer Interface Triggering an Active Ankle–Foot Orthosis for Inducing Cortical Neural Plasticity , 2014, IEEE Transactions on Biomedical Engineering.

[14]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[15]  Arash Mohammadi,et al.  Feature Space Reduction for Single Trial EEG Classification based on Wavelet Decomposition , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Cuntai Guan,et al.  A Unified Fisher’s Ratio Learning Method for Spatial Filter Optimization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Arash Mohammadi,et al.  Ternary ECOC classifiers coupled with optimized spatio-spectral patterns for multiclass motor imagery classification , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[18]  Xinjun Sheng,et al.  A Stimulus-Independent Hybrid BCI Based on Motor Imagery and Somatosensory Attentional Orientation , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Bin He,et al.  EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks , 2016, IEEE Transactions on Biomedical Engineering.

[20]  Haiping Lu,et al.  Regularized Common Spatial Pattern With Aggregation for EEG Classification in Small-Sample Setting , 2010, IEEE Transactions on Biomedical Engineering.

[21]  Xiaofeng Xie,et al.  Motor Imagery Classification Based on Bilinear Sub-Manifold Learning of Symmetric Positive-Definite Matrices , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[23]  Arash Mohammadi,et al.  Bayesian Optimized Spectral Filters Coupled With Ternary ECOC for Single-Trial EEG Classification , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[25]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  José del R. Millán,et al.  Towards Independence: A BCI Telepresence Robot for People With Severe Motor Disabilities , 2015, Proceedings of the IEEE.

[27]  Arash Mohammadi,et al.  A Bayesian Framework to Optimize Double Band Spectra Spatial Filters for Motor Imagery Classification , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[29]  Sebastian Stober,et al.  Learning discriminative features from electroencephalography recordings by encoding similarity constraints , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  Zhilin Zhang,et al.  Evolving Signal Processing for Brain–Computer Interfaces , 2012, Proceedings of the IEEE.

[31]  Rita Cucchiara,et al.  From Depth Data to Head Pose Estimation: A Siamese Approach , 2017, VISIGRAPP.

[32]  Moritz Grosse-Wentrup,et al.  Using brain–computer interfaces to induce neural plasticity and restore function , 2011, Journal of neural engineering.

[33]  Emanuele Maiorana,et al.  EEG-Based Biometric Verification Using Siamese CNNs , 2019, ICIAP Workshops.

[34]  Clement Hamani,et al.  Reconstruction of reaching movement trajectories using electrocorticographic signals in humans , 2017, PloS one.