A novel motor imagery hybrid brain computer interface using EEG and functional transcranial Doppler ultrasound

BACKGROUND Hybrid brain computer interfaces (BCIs) combining multiple brain imaging modalities have been proposed recently to boost the performance of single modality BCIs. NEW METHOD In this paper, we propose a novel motor imagery (MI) hybrid BCI that uses electrical brain activity recorded using Electroencephalography (EEG) as well as cerebral blood flow velocity measured using functional transcranial Doppler ultrasound (fTCD). Features derived from the power spectrum for both EEG and fTCD signals were calculated. Mutual information and linear support vector machines (SVM) were employed for feature selection and classification. RESULTS Using the EEG-fTCD combination, average accuracies of 88.33%, 89.48%, and 82.38% were achieved for right arm MI versus baseline, left arm MI versus baseline, and right arm MI versus left arm MI respectively. Compared to performance measures obtained using EEG only, the hybrid system provided significant improvement in terms of accuracy by 4.48%, 5.36%, and 4.76% respectively. In addition, average transmission rates of 4.17, 5.45, and 10.57 bits/min were achieved for right arm MI versus baseline, left arm MI versus baseline, and right arm MI versus left arm MI respectively. COMPARISON WITH EXISTING METHODS Compared to EEG-fNIRS hybrid BCIs in literature, we achieved similar or higher accuracies with shorter task duration. CONCLUSIONS The proposed hybrid system is a promising candidate for real-time BCI applications.

[1]  D F Hanley,et al.  Validation of transcranial Doppler ultrasound with a stereotactic neurosurgical technique. , 1995, Journal of neurosurgery.

[2]  Tom Chau,et al.  A Brain-Computer Interface Based on Bilateral Transcranial Doppler Ultrasound , 2011, PloS one.

[3]  Murat Akcakaya,et al.  A brain-computer interface based on functional transcranial doppler ultrasound using wavelet transform and support vector machines , 2018, Journal of Neuroscience Methods.

[4]  Jonathan R Wolpaw,et al.  Brain–computer interface systems: progress and prospects , 2007, Expert review of medical devices.

[5]  Hasan Onur Keles,et al.  Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks , 2016, PloS one.

[6]  Motoaki Kawanabe,et al.  Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information , 2014, NeuroImage.

[7]  Yuanqing Li,et al.  Control of a Wheelchair in an Indoor Environment Based on a Brain–Computer Interface and Automated Navigation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Brock Wester,et al.  Flight simulation using a Brain-Computer Interface: A pilot, pilot study , 2017, Experimental Neurology.

[9]  Thomas Hofmann,et al.  A brain computer interface with online feedback based on magnetoencephalography , 2005, ICML.

[10]  L Petrosini,et al.  Cerebral blood flow velocity changes during meaningful and meaningless gestures – a functional transcranial Doppler study , 2006, European journal of neurology.

[11]  David E. J. Linden,et al.  Real-time fMRI brain-computer interface: development of a “motivational feedback” subsystem for the regulation of visual cue reactivity , 2014, Front. Behav. Neurosci..

[12]  Seungjin Choi,et al.  A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery , 2015, Journal of Neuroscience Methods.

[13]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[14]  Yunfa Fu,et al.  A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching , 2015, Journal of neural engineering.

[15]  G A Clark,et al.  Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves , 2016, Journal of neural engineering.

[16]  Keum-Shik Hong,et al.  fNIRS-based brain-computer interfaces: a review , 2015, Front. Hum. Neurosci..

[17]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[18]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[19]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Chun-Yi Su,et al.  Brain–Machine Interface and Visual Compressive Sensing-Based Teleoperation Control of an Exoskeleton Robot , 2017, IEEE Transactions on Fuzzy Systems.

[21]  T. Ward,et al.  Brain computer interfaces for neurorehabilitation – its current status as a rehabilitation strategy post-stroke. , 2015, Annals of physical and rehabilitation medicine.

[22]  Byoung-Kyong Min,et al.  Neuroimaging-based approaches in the brain-computer interface. , 2010, Trends in biotechnology.

[23]  Tom Chau,et al.  Pattern classification to optimize the performance of Transcranial Doppler Ultrasonography-based brain machine interface , 2015, Pattern Recognit. Lett..

[24]  J. J. Higgins,et al.  A Comparison of the Power of Wilcoxon's Rank-Sum Statistic to that of Student'st Statistic Under Various Nonnormal Distributions , 1980 .

[25]  Tom Chau,et al.  Towards a hemodynamic BCI using transcranial Doppler without user-specific training data. , 2013, Journal of neural engineering.

[26]  Jae Gwan Kim,et al.  Recent functional near infrared spectroscopy based brain computer interface systems: Developments, applications and challenges , 2014 .

[27]  R. Vlek,et al.  Combined EEG-fNIRS Decoding of Motor Attempt and Imagery for Brain Switch Control: An Offline Study in Patients With Tetraplegia , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Gert Pfurtscheller,et al.  Characterization of four-class motor imagery EEG data for the BCI-competition 2005 , 2005, Journal of neural engineering.

[29]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[30]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[31]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[32]  Guy Vingerhoets,et al.  Transcranial Doppler Ultrasonography Monitoring of Cerebral Hemodynamics During Performance of Cognitive Tasks: A Review , 2000, Neuropsychology Review.

[33]  W. Tyler,et al.  Transcranial Focused Ultrasound Modulates Intrinsic and Evoked EEG Dynamics , 2014, Brain Stimulation.

[34]  A. U. Pehlivan,et al.  Current Trends in Robot-Assisted Upper-Limb Stroke Rehabilitation: Promoting Patient Engagement in Therapy , 2014, Current Physical Medicine and Rehabilitation Reports.

[35]  Brendan Z. Allison,et al.  A feasibility study on SSVEP-based interaction with motivating and immersive virtual and augmented reality , 2017, ArXiv.

[36]  Manfred Kaps,et al.  Practice Standards for Transcranial Doppler Ultrasound: Part I—Test Performance , 2007, Journal of neuroimaging : official journal of the American Society of Neuroimaging.

[37]  Cuntai Guan,et al.  EEG-Based Strategies to Detect Motor Imagery for Control and Rehabilitation , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[38]  Pablo A. Estévez,et al.  A review of feature selection methods based on mutual information , 2013, Neural Computing and Applications.

[39]  Cuntai Guan,et al.  Clinical study of neurorehabilitation in stroke using EEG-based motor imagery brain-computer interface with robotic feedback , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.