Analysis and classification of hybrid BCI based on motor imagery and speech imagery

Abstract As a new communication mode between human and computers, brain-computer interfaces (BCIs) based on electroencephalography (EEG) have been widely studied. In order to increase the operational dimension of BCIs, this paper proposes a hybrid BCI based on motor imagery and speech imagery. According to one versus one calculation model, common spatial pattern (CSP) is extended to a three-category algorithm. The energy eigenvalues extracted by CSP are combined with synchronization eigenvalues, which are respectively calculated by cross-correlation function and phase locking value (PLV). The maximum difference of synchronization between two mental tasks is considered as a method to confirm the channel pairs. The experimental results of ten subjects show that: the highest average classification accuracy in three mental tasks is speech imagery (74.3%); imagining left hand movement is followed (71.4%); imagining right hand movement is the last one (69.8%). The classification results can be improved by combining synchronization and CSP, and the synchronization from cross-correlation function is better than PLV. The frequency range and channel pairs both have certain individual differences. By individually adjusting the appropriate settings for each subject, the usage efficiency of BCIs can be improved.

[1]  Victoria Peterson,et al.  A penalized time-frequency band feature selection and classification procedure for improved motor intention decoding in multichannel EEG , 2019, Journal of neural engineering.

[2]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[3]  Yu Zhang,et al.  Analysis and classification of speech imagery EEG for BCI , 2013, Biomed. Signal Process. Control..

[4]  Bin He,et al.  Brain–Computer Interfaces Using Sensorimotor Rhythms: Current State and Future Perspectives , 2014, IEEE Transactions on Biomedical Engineering.

[5]  Ting Li,et al.  Decoding Voluntary Movement of Single Hand Based on Analysis of Brain Connectivity by Using EEG Signals , 2018, Front. Hum. Neurosci..

[6]  Farhad Faradji,et al.  Toward development of a two-state brain–computer interface based on mental tasks , 2011, Journal of neural engineering.

[7]  Li Wang,et al.  Improvement of mental tasks with relevant speech imagery for brain-computer interfaces , 2016 .

[8]  Panagiotis G. Simos,et al.  A comparison of functional MRI and magnetoencephalography for receptive language mapping , 2007, Journal of Neuroscience Methods.

[9]  Asdrúbal López Chau,et al.  Convex and concave hulls for classification with support vector machine , 2013, Neurocomputing.

[10]  Athanasios V. Vasilakos,et al.  Brain computer interface: control signals review , 2017, Neurocomputing.

[11]  Cuntai Guan,et al.  A Unified Fisher’s Ratio Learning Method for Spatial Filter Optimization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Yijun Wang,et al.  Amplitude and phase coupling measures for feature extraction in an EEG-based brain–computer interface , 2007, Journal of neural engineering.

[13]  R. K. Agrawal,et al.  Relevant Feature Selection from a Combination of Spectral-Temporal and Spatial Features for Classification of Motor Imagery EEG , 2018, Journal of Medical Systems.

[14]  Vera Kaiser,et al.  Thinking Penguin: Multimodal Brain–Computer Interface Control of a VR Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[15]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[16]  Lei Sun,et al.  A contralateral channel guided model for EEG based motor imagery classification , 2018, Biomed. Signal Process. Control..

[17]  Yuanqing Li,et al.  Target Selection With Hybrid Feature for BCI-Based 2-D Cursor Control , 2012, IEEE Transactions on Biomedical Engineering.

[18]  Ioannis Kompatsiaris,et al.  Exploiting the heightened phase synchrony in patients with neuromuscular disease for the establishment of efficient motor imagery BCIs , 2018, Journal of NeuroEngineering and Rehabilitation.

[19]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[20]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[21]  Aime Lay-Ekuakille,et al.  Experimental characterization of an implantable neuro-packaging for EEG signal recording and measurement , 2016 .

[22]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Dheeraj Sharma,et al.  Features based on analytic IMF for classifying motor imagery EEG signals in BCI applications , 2018 .

[24]  Wei Wu,et al.  Deep learning based on Batch Normalization for P300 signal detection , 2018, Neurocomputing.

[25]  C. Neuper,et al.  Long-term evaluation of a 4-class imagery-based brain–computer interface , 2013, Clinical Neurophysiology.

[26]  Eric Leuthardt,et al.  Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition , 2011, NeuroImage.

[27]  Sungho Jo,et al.  A Low-Cost EEG System-Based Hybrid Brain-Computer Interface for Humanoid Robot Navigation and Recognition , 2013, PloS one.

[28]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[29]  Turker Tekin Erguzel,et al.  Analysis of Time - Frequency EEG Feature Extraction Methods for Mental Task Classification , 2017, Int. J. Comput. Intell. Syst..

[30]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[31]  R Quian Quiroga,et al.  Performance of different synchronization measures in real data: a case study on electroencephalographic signals. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[32]  Klaus-Robert Müller,et al.  Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms , 2004, IEEE Transactions on Biomedical Engineering.

[33]  Jie Li,et al.  Covert Verb Reading Contributes to Signal Classification of Motor Imagery in BCI , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[35]  Yongtian He,et al.  Deep learning for electroencephalogram (EEG) classification tasks: a review , 2019, Journal of neural engineering.

[36]  Yue Zhao,et al.  A Wireless BCI and BMI System for Wearable Robots , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[38]  Klaus-Robert Müller,et al.  A large scale screening study with a SMR-based BCI: Categorization of BCI users and differences in their SMR activity , 2019, PloS one.

[39]  Panagiotis Artemiadis,et al.  Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features , 2018, Journal of neural engineering.

[40]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[41]  Hongzhi Qi,et al.  A novel technique for phase synchrony measurement from the complex motor imaginary potential of combined body and limb action , 2010, Journal of neural engineering.

[42]  R. Ramírez-Mendoza,et al.  Motor imagery based brain–computer interfaces: An emerging technology to rehabilitate motor deficits , 2015, Neuropsychologia.

[43]  Masashi Sugiyama,et al.  Importance-weighted covariance estimation for robust common spatial pattern , 2014, Pattern Recognit. Lett..

[44]  Clemens Brunner,et al.  Nonstationary Brain Source Separation for Multiclass Motor Imagery , 2010, IEEE Transactions on Biomedical Engineering.

[45]  Jinchang Ren,et al.  EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges , 2019, Sensors.

[46]  Edward F. Chang,et al.  Speech synthesis from neural decoding of spoken sentences , 2019, Nature.

[47]  Nathan E. Crone,et al.  The Potential for a Speech Brain–Computer Interface Using Chronic Electrocorticography , 2019, Neurotherapeutics.

[48]  Hua Wang,et al.  Detection of motor imagery EEG signals employing Naïve Bayes based learning process , 2016 .

[49]  Reza Abiri,et al.  A comprehensive review of EEG-based brain–computer interface paradigms , 2019, Journal of neural engineering.

[50]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.