A Deep Learning Framework for Decoding Motor Imagery Tasks of the Same Hand Using EEG Signals

This study aims to increase the control’s dimensions of the electroencephalography (EEG)-based brain-computer interface (BCI) systems by distinguishing between the motor imagery (MI) tasks associated with fine body-parts of the same hand, such as the wrist and fingers. This in turn can enable individuals who are suffering from transradial amputations to better control prosthetic hands and to perform various dexterous hand tasks. In particular, we present a novel three-stage framework for decoding MI tasks of the same hand. The three stages of the proposed framework are the input, feature extraction, and classification stages. At the input stage, we employ a quadratic time-frequency distribution (QTFD) to analyze the EEG signals in the joint time-frequency domain. The use of a QTFD enables to transform the EEG signals into a set of two-dimensional (2D) time-frequency images (TFIs) that describe the distribution of the energy encapsulated within the EEG signals in terms of the time, frequency, and electrode position. At the feature extraction stage, we design a new convolutional neural network (CNN) architecture that can automatically analyze and extract salient features from the TFIs created at the input stage. Finally, the features obtained at the feature extraction stage are passed to the classification stage to assign each input TFI to one of the eleven MI tasks that are considered in the current study. The performance of our proposed framework is evaluated using EEG signals that were acquired from eighteen able-bodied subjects and four transradial amputated subjects while performing eleven MI tasks within the same hand. The average classification accuracies obtained for the able-bodied and transradial amputated subjects are 73.7% and 72.8%, respectively. Moreover, our proposed framework yields 14.5% and 11.2% improvements over the results obtained for the able-bodied and transradial amputated subjects, respectively, using conventional QTFD-based handcrafted features and a multi-class support vector machine classifier. The results demonstrate the efficacy of the proposed framework to decode the MI tasks associated with the same hand for able-bodied and transradial amputated subjects.

[1]  Klaus-Robert Müller,et al.  The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials , 2004, IEEE Transactions on Biomedical Engineering.

[2]  Boualem Boashash,et al.  Time-Frequency Signal Analysis and Processing: A Comprehensive Reference , 2015 .

[3]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[4]  M. Jeannerod.  Mental imagery in the motor context , 1995, Neuropsychologia.

[5]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[6]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[7]  Mohammad I. Daoud,et al.  EEG-based tonic cold pain recognition system using wavelet transform , 2017, Neural Computing and Applications.

[8]  L. Resnik,et al.  Advanced upper limb prosthetic devices: implications for upper limb prosthetic rehabilitation. , 2012, Archives of physical medicine and rehabilitation.

[9]  W. De Clercq,et al.  Automatic Removal of Ocular Artifacts in the EEG without an EOG Reference Channel , 2006, Proceedings of the 7th Nordic Signal Processing Symposium - NORSIG 2006.

[10]  J. O. Toole.  Discrete quadratic time-frequency distributions: Definition, computation, and a newborn electroencephalogram application , 2009 .

[11]  Moritz Grosse-Wentrup,et al.  Multiclass Common Spatial Patterns and Information Theoretic Feature Extraction , 2008, IEEE Transactions on Biomedical Engineering.

[12]  J L Contreras-Vidal,et al.  Multisession, noninvasive closed-loop neuroprosthetic control of grasping by upper limb amputees. , 2016, Progress in brain research.

[13]  Chao Li,et al.  A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control , 2016, Sensors.

[14]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[15]  Carlo Menon,et al.  EEG Classification of Different Imaginary Movements within the Same Limb , 2015, PloS one.

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  Mohammad I. Daoud,et al.  EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution , 2018, Sensors.

[18]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[20]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[21]  William J. Williams,et al.  Improved time-frequency representation of multicomponent signals using exponential kernels , 1989, IEEE Trans. Acoust. Speech Signal Process..

[22]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[23]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[24]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[25]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[26]  Francisco Sepulveda,et al.  Classifying mental tasks based on features of higher-order statistics from EEG signals in brain-computer interface , 2008, Inf. Sci..

[27]  Mohammad I. Daoud,et al.  EEG-Based Brain-Computer Interface for Decoding Motor Imagery Tasks within the Same Hand Using Choi-Williams Time-Frequency Distribution , 2017, Sensors.

[28]  Bin He,et al.  EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks , 2016, IEEE Transactions on Biomedical Engineering.

[29]  Mohammad I. Daoud,et al.  EEG-based BCI system for decoding finger movements within the same hand , 2019, Neuroscience Letters.

[30]  Tingxi Wen,et al.  Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals , 2018, IEEE Access.

[31]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[32]  Ke Liao,et al.  Decoding Individual Finger Movements from One Hand Using Human EEG Signals , 2014, PloS one.

[33]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[34]  U. Rajendra Acharya,et al.  Deep learning for healthcare applications based on physiological signals: A review , 2018, Comput. Methods Programs Biomed..

[35]  Hans-Jochen Heinze,et al.  Single trial discrimination of individual finger movements on one hand: A combined MEG and EEG study , 2012, NeuroImage.

[36]  Francisco Sepulveda,et al.  Delta band contribution in cue based single trial classification of real and imaginary wrist movements , 2008, Medical & Biological Engineering & Computing.

[37]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[38]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[39]  L. Cohen,et al.  Time-frequency distributions-a review , 1989, Proc. IEEE.

[40]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[41]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human EEG , 2017, ArXiv.

[42]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[43]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Sadasivan Puthusserypady,et al.  An end-to-end deep learning approach to MI-EEG signal classification for BCIs , 2018, Expert Syst. Appl..

[45]  G. Buccino,et al.  Action observation versus motor imagery in learning a complex motor task: A short review of literature and a kinematics study , 2013, Neuroscience Letters.

[46]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.