An end-to-end deep learning approach to MI-EEG signal classification for BCIs

Abstract Goal: To develop and implement a Deep Learning (DL) approach for an electroencephalogram (EEG) based Motor Imagery (MI) Brain-Computer Interface (BCI) system that could potentially be used to improve the current stroke rehabilitation strategies. Method: The DL model is using Convolutional Neural Network (CNN) layers for learning generalized features and dimension reduction, while a conventional Fully Connected (FC) layer is used for classification. Together they build a unified end-to-end model that can be applied to raw EEG signals. This previously proposed model was applied to a new set of data to validate its robustness against data variations. Furthermore, it was extended by subject-specific adaptation. Lastly, an analysis of the learned filters provides insights into how such a model derives a classification decision. Results: The selected global classifier reached 80.38%, 69.82%, and 58.58% mean accuracies for datasets with two, three, and four classes, respectively, validated using 5-fold crossvalidation. As a novel approach in this context, transfer learning was used to adapt the global classifier to single individuals improving the overall mean accuracy to 86.49%, 79.25%, and 68.51%, respectively. The global models were trained on 3s segments of EEG data from different subjects than they were tested on, which proved the generalization performance of the model. Conclusion: The results are comparable with the reported accuracy values in related studies and the presented model outperforms the results in the literature on the same underlying data. Given that the model can learn features from data without having to use specialized feature extraction methods, DL should be considered as an alternative to established EEG classification methods, if enough data is available.

[1]  Vivek Prabhakaran,et al.  BCI-FES: could a new rehabilitation device hold fresh promise for stroke patients? , 2014, Expert review of medical devices.

[2]  Michael Brainin,et al.  Role of repetitive transcranial magnetic stimulation in stroke rehabilitation. , 2013, Frontiers of neurology and neuroscience.

[3]  An H. Do,et al.  Brain-Computer Interface Controlled Functional Electrical Stimulation System for Ankle Movement , 2011, Journal of NeuroEngineering and Rehabilitation.

[4]  Gerardo G. Gentiletti,et al.  BCI-FES system for neuro-rehabilitation of stroke patients , 2016 .

[5]  Franjo Jović,et al.  CLASSIFICATION OF WAVELET TRANSFORMED EEG SIGNALS WITH NEURAL NETWORK FOR IMAGINED MENTAL AND MOTOR TASKS , 2013 .

[6]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[7]  Danilo P. Mandic,et al.  Augmented Complex Common Spatial Patterns for Classification of Noncircular EEG From Motor Imagery Tasks , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Yijun Wang,et al.  Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[9]  G. Schalk,et al.  Evolution of brain-computer interfaces : going beyond classic motor physiology , 2009 .

[10]  Bin He,et al.  Motor imagery task classification for brain computer interface applications using spatiotemporal principle component analysis , 2004, Neurological research.

[11]  Tatsuhiko Tsunoda,et al.  A Deep Learning Approach for Motor Imagery EEG Signal Classification , 2016, 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE).

[12]  Wolfgang Rosenstiel,et al.  Online Artifact Removal for Brain-Computer Interfaces Using Support Vector Machines and Blind Source Separation , 2007, Comput. Intell. Neurosci..

[13]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[14]  Isabelle Bloch,et al.  Subject-Specific Channel Selection Using Time Information for Motor Imagery Brain–Computer Interfaces , 2016, Cognitive Computation.

[15]  Vinod Achutavarrier Prasad,et al.  Optimized Bi-Objective EEG Channel Selection and Cross-Subject Generalization With Brain–Computer Interfaces , 2016, IEEE Transactions on Human-Machine Systems.

[16]  Zoran Nenadic,et al.  Brain-computer interface controlled functional electrical stimulation device for foot drop due to stroke , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[18]  Jennifer Rodger,et al.  Repetitive transcranial magnetic stimulation for stroke rehabilitation-potential therapy or misplaced hope? , 2015, Restorative neurology and neuroscience.

[19]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[20]  Suk-Tak Chan,et al.  BCI-FES training system design and implementation for rehabilitation of stroke patients , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[21]  M. Nuttin,et al.  A brain-actuated wheelchair: Asynchronous and non-invasive Brain–computer interfaces for continuous control of robots , 2008, Clinical Neurophysiology.

[22]  Ana Loboda,et al.  Discrimination of EEG-Based Motor Imagery Tasks by Means of a Simple Phase Information Method , 2014 .

[23]  Qingshan She,et al.  Classification of Motor Imagery EEG Signals with Support Vector Machines and Particle Swarm Optimization , 2016, Comput. Math. Methods Medicine.

[24]  M. Molinari,et al.  Brain–computer interface boosts motor imagery practice during stroke recovery , 2015, Annals of neurology.

[25]  Christoph Stippich,et al.  Somatotopic mapping of the human primary sensorimotor cortex during motor imagery and motor execution by functional magnetic resonance imaging , 2002, Neuroscience Letters.

[26]  Danilo P. Mandic,et al.  Motor Imagery Classification Using Mu and Beta Rhythms of EEG with Strong Uncorrelating Transform Based Complex Common Spatial Patterns , 2016, Comput. Intell. Neurosci..

[27]  T. Mulder Motor imagery and action observation: cognitive tools for rehabilitation , 2007, Journal of Neural Transmission.

[28]  Konstantinos N. Plataniotis,et al.  Separable Common Spatio-Spectral Patterns for Motor Imagery BCI Systems , 2016, IEEE Transactions on Biomedical Engineering.

[29]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[30]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[31]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[32]  Isabelle Bloch,et al.  Subject-specific time-frequency selection for multi-class motor imagery-based BCIs using few Laplacian EEG channels , 2017, Biomed. Signal Process. Control..

[33]  Cheolsoo Park,et al.  Classification of Motor Imagery BCI Using Multivariate Empirical Mode Decomposition , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[35]  Isabelle Bloch,et al.  Time-frequency optimization for discrimination between imagination of right and left hand movements based on two bipolar electroencephalography channels , 2014, EURASIP Journal on Advances in Signal Processing.

[36]  Moritz Grosse-Wentrup,et al.  Multiclass Common Spatial Patterns and Information Theoretic Feature Extraction , 2008, IEEE Transactions on Biomedical Engineering.

[37]  Djamel Bouchaffra,et al.  Random Forest and Filter Bank Common Spatial Patterns for EEG-Based Motor Imagery Classification , 2014, 2014 5th International Conference on Intelligent Systems, Modelling and Simulation.

[38]  Cuntai Guan,et al.  High performance P300 speller for brain-computer interface , 2004, IEEE International Workshop on Biomedical Circuits and Systems, 2004..

[39]  Cuntai Guan,et al.  On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[40]  N. Birbaumer,et al.  Brain–computer interfaces for communication and rehabilitation , 2016, Nature Reviews Neurology.

[41]  Hongtao Lu,et al.  Classification of Motor Imagery EEG Signals with Deep Learning Models , 2017, IScIDE.

[42]  Ronald Davis,et al.  Neural networks and deep learning , 2017 .