Deep Channel-Correlation Network for Motor Imagery Decoding From the Same Limb

Motor imagery (MI) is an important brain-computer interface (BCI) paradigm, which can be applied without external stimulus. Imagining different joint movements from the same limb allows intuitive control of the outer devices. However, few researches focused on this field, and the decoding accuracy limited the applications for practical use. In this study, we aim to use deep learning methods to explore the ceiling of the decoding performance of three tasks: the resting state, the MI of right hand and right elbow. To represent the brain functional relationships, the correlation matrix that consists of correlation coefficients between electrodes (channels) was calculated as features. We proposed the Channel-Correlation Network to learn the overall representation among channels for classification. Ensemble learning was applied to integrate the output of multiple Channel-Correlation Networks. Our proposed method achieved the decoding accuracy of up to 87.03% in the 3-class scenario. The results demonstrated the effectiveness of deep learning method for decoding MI of different joints from the same limb and the potential of this fine paradigm to be applied in practice.

[1]  Ruth Dickstein,et al.  Motor Imagery Training for Gait Rehabilitation of People With Post-Stroke Hemiparesis: Practical Applications and Protocols , 2018, Global Journal of Health Science.

[2]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[3]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[4]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[5]  Kyuwan Choi,et al.  Control of a Wheelchair by Motor Imagery in Real Time , 2008, IDEAL.

[6]  J. Shaw.  Correlation and coherence analysis of the EEG: a selective tutorial review. , 1984, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  Carlo Menon,et al.  EEG Classification of Different Imaginary Movements within the Same Limb , 2015, PloS one.

[8]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[11]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[12]  Deniz Erdogmus,et al.  Decoding complex imagery hand gestures , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Carlo Menon,et al.  Evaluating the versatility of EEG models generated from motor imagery tasks: An exploratory investigation on upper-limb elbow-centered motor imagery tasks , 2017, PloS one.

[14]  Keng Peng Tee,et al.  EEG-Based Classification of Fast and Slow Hand Movements Using Wavelet-CSP Algorithm , 2013, IEEE Transactions on Biomedical Engineering.

[15]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[16]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[17]  Effie Chew,et al.  Is Motor‐Imagery Brain‐Computer Interface Feasible in Stroke Rehabilitation? , 2014, PM & R : the journal of injury, function, and rehabilitation.

[18]  Lina Yao,et al.  Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface , 2017, AAAI.

[19]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[20]  Cuntai Guan,et al.  A Randomized Controlled Trial of EEG-Based Motor Imagery Brain-Computer Interface Robotic Rehabilitation for Stroke , 2015, Clinical EEG and neuroscience.

[21]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[23]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[24]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[25]  W. De Clercq,et al.  Automatic Removal of Ocular Artifacts in the EEG without an EOG Reference Channel , 2006, Proceedings of the 7th Nordic Signal Processing Symposium - NORSIG 2006.

[26]  Chao Wu,et al.  DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Pheng-Ann Heng,et al.  Multiclass support matrix machine for single trial EEG classification , 2018, Neurocomputing.

[28]  Carlo Menon,et al.  Classifying three imaginary states of the same upper extremity using time-domain features , 2017, PloS one.

[29]  Yufei Huang,et al.  Prediction of fatigue-related driver performance from EEG data by deep Riemannian model , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  Zhaoxiang Zhang,et al.  Hierarchical Convolutional Neural Networks for EEG-Based Emotion Recognition , 2017, Cognitive Computation.

[31]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[32]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[33]  Dong Ming,et al.  Evaluation of EEG Oscillatory Patterns and Cognitive Process during Simple and Compound Limb Motor Imagery , 2014, PloS one.

[34]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.

[35]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[36]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[37]  T.M. McGinnity,et al.  Comparative Analysis of Spectral Approaches to Feature Extraction for EEG-Based Motor Imagery Classification , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[38]  S. Makeig.  Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. , 1993, Electroencephalography and clinical neurophysiology.

[39]  Stanislas Chambon,et al.  A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  Bin He,et al.  EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks , 2016, IEEE Transactions on Biomedical Engineering.

[41]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[42]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[43]  Yufei Huang,et al.  Driver's fatigue prediction by deep covariance learning from EEG , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[44]  Gernot R. Müller-Putz,et al.  Control of an Electrical Prosthesis With an SSVEP-Based BCI , 2008, IEEE Transactions on Biomedical Engineering.

[45]  Gerald V. Smith,et al.  Comparing brain activation associated with isolated upper and lower limb movement across corresponding joints , 2002, Human brain mapping.

[46]  Dean J Krusienski,et al.  Brain-computer interfaces in medicine. , 2012, Mayo Clinic proceedings.

[47]  M. Bergamasco,et al.  A New Gaze-BCI-Driven Control of an Upper Limb Exoskeleton for Rehabilitation in Real-World Tasks , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[48]  Changde Du,et al.  Improving EEG-Based Motor Imagery Classification via Spatial and Temporal Recurrent Neural Networks , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[49]  Hongzhi Qi,et al.  EEG feature comparison and classification of simple and compound limb motor imagery , 2013, Journal of NeuroEngineering and Rehabilitation.

[50]  Mohammad I. Daoud,et al.  EEG-Based Brain-Computer Interface for Decoding Motor Imagery Tasks within the Same Hand Using Choi-Williams Time-Frequency Distribution , 2017, Sensors.

[51]  Francisco Sepulveda,et al.  Delta band contribution in cue based single trial classification of real and imaginary wrist movements , 2008, Medical & Biological Engineering & Computing.

[52]  Pasin Israsena,et al.  Comparison of EEG measurement of upper limb movement in motor imagery training system , 2018, BioMedical Engineering OnLine.

[53]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[54]  Na Lu,et al.  A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[55]  Joelle Pineau,et al.  Learning Robust Features using Deep Learning for Automatic Seizure Detection , 2016, MLHC.