Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks

Non-invasive Brain-Computer Interfaces (BCI) have demonstrated great promise for neuroprosthetics and assistive devices. Here we aim to investigate methods to combine Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) in an asynchronous Sensory Motor rhythm (SMR)-based BCI. We attempted to classify 4 different executed movements, namely, Right-Arm—Left-Arm—Right-Hand—Left-Hand tasks. Previous studies demonstrated the benefit of EEG-fNIRS combination. However, since normally fNIRS hemodynamic response shows a long delay, we investigated new features, involving slope indicators, in order to immediately detect changes in the signals. Moreover, Common Spatial Patterns (CSPs) have been applied to both EEG and fNIRS signals. 15 healthy subjects took part in the experiments and since 25 trials per class were available, CSPs have been regularized with information from the entire population of participants and optimized using genetic algorithms. The different features have been compared in terms of performance and the dynamic accuracy over trials shows that the introduced methods diminish the fNIRS delay in the detection of changes.

[1]  David A. Boas,et al.  A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain Activation , 2002, NeuroImage.

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  David G. Stork,et al.  Pattern Classification , 1973 .

[4]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[5]  Erno J. Hermans,et al.  Enhanced sensitivity with fast three‐dimensional blood‐oxygen‐level‐dependent functional MRI: comparison of SENSE–PRESTO and 2D‐EPI at 3 T , 2008, NMR in biomedicine.

[6]  Gabriel Curio,et al.  Speeding up classification of multi-channel brain-computer interfaces: common spatial patterns for slow cortical potentials , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[7]  J. Mazziotta,et al.  Mapping motor representations with positron emission tomography , 1994, Nature.

[8]  E. Halgren,et al.  Motor-cortical activity in tetraplegics , 2001, Nature.

[9]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[10]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[11]  H. Noordmans,et al.  Development of a functional magnetic resonance imaging protocol for intraoperative localization of critical temporoparietal language areas , 2002, Annals of neurology.

[12]  Keum-Shik Hong,et al.  Classification of functional near-infrared spectroscopy signals corresponding to the right- and left-wrist motor imagery for development of a brain–computer interface , 2013, Neuroscience Letters.

[13]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[14]  Steven C. Cramer,et al.  Brain activation during execution and motor imagery of novel and skilled sequential hand movements , 2005, NeuroImage.

[15]  W. Singer,et al.  Hemodynamic Signals Correlate Tightly with Synchronized Gamma Oscillations , 2005, Science.

[16]  A. Yodh,et al.  Diffuse optics for tissue monitoring and tomography , 2010, Reports on progress in physics. Physical Society.

[17]  Brendan Z. Allison,et al.  Brain-Computer Interfaces: A Gentle Introduction , 2009 .

[18]  I. Toni,et al.  Motor imagery: A window into the mechanisms and alterations of the motor system , 2008, Cortex.

[19]  Martin Wolf,et al.  A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology , 2014, NeuroImage.

[20]  E. Naito,et al.  Internally Simulated Movement Sensations during Motor Imagery Activate Cortical Motor Areas and the Cerebellum , 2002, The Journal of Neuroscience.

[21]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.

[22]  K. Müller,et al.  Single Trial Classification of Motor Imagination Using 6 Dry EEG Electrodes , 2007, PloS one.

[23]  Wei-Yen Hsu,et al.  EEG-based motor imagery analysis using weighted wavelet transform features , 2009, Journal of Neuroscience Methods.

[24]  Bin He,et al.  Cortical Imaging of Event-Related (de)Synchronization During Online Control of Brain-Computer Interface Using Minimum-Norm Estimates in Frequency Domain , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Julien Doyon,et al.  Functional neuroanatomical networks associated with expertise in motor imagery , 2008, NeuroImage.

[26]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[27]  Alan C. Evans,et al.  Motor Learning Produces Parallel Dynamic Functional Changes during the Execution and Imagination of Sequential Foot Movements , 2002, NeuroImage.

[28]  Matthew J. Brookes,et al.  GLM-beamformer method demonstrates stationary field, alpha ERD and gamma ERS co-localisation with fMRI BOLD response in visual cortex , 2005, NeuroImage.

[29]  O. Bertrand,et al.  Relationship between task‐related gamma oscillations and BOLD signal: New insights from combined fMRI and intracranial EEG , 2007, Human brain mapping.

[30]  Zhongming Liu,et al.  An enhanced time-frequency-spatial approach for motor imagery classification , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  A. Villringer,et al.  Rolandic alpha and beta EEG rhythms' strengths are inversely related to fMRI‐BOLD signal in primary somatosensory and motor cortex , 2009, Human brain mapping.

[32]  J B Poline,et al.  Partially overlapping neural networks for real and imagined hand movements. , 2000, Cerebral cortex.

[33]  Bin He,et al.  Electrophysiological Imaging of Brain Activity and Connectivity—Challenges and Opportunities , 2011, IEEE Transactions on Biomedical Engineering.

[34]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[35]  Ivan Toni,et al.  Neural Topography and Content of Movement Representations , 2005, Journal of Cognitive Neuroscience.

[36]  Haiping Lu,et al.  Regularized common spatial patterns with generic learning for EEG signal classification , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[37]  Scott T. Grafton,et al.  Selective Activation of a Parietofrontal Circuit during Implicitly Imagined Prehension , 2002, NeuroImage.

[38]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[39]  Nick F. Ramsey,et al.  Automated electrocorticographic electrode localization on individually rendered brain surfaces , 2010, Journal of Neuroscience Methods.

[40]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[41]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[42]  M. Hallett,et al.  Cerebral Processes Related to Visuomotor Imagery and Generation of Simple Finger Movements Studied with Positron Emission Tomography , 1998, NeuroImage.

[43]  Gernot R. Müller-Putz,et al.  Discrimination of Motor Imagery-Induced EEG Patterns in Patients with Complete Spinal Cord Injury , 2009, Comput. Intell. Neurosci..

[44]  Ryuta Kawashima,et al.  A NIRS-based brain-computer interface system during motor imagery: System development and online feedback training , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[45]  A. Sirigu,et al.  The Mental Representation of Hand Movements After Parietal Cortex Damage , 1996, Science.

[46]  M. Diamond,et al.  Primary Motor and Sensory Cortex Activation during Motor Performance and Motor Imagery: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[47]  Bin He,et al.  Negative covariation between task-related responses in alpha/beta-band activity and BOLD in human sensorimotor cortex: An EEG and fMRI study of motor imagery and movements , 2010, NeuroImage.

[48]  Bin He,et al.  INSTITUTE OF PHYSICS PUBLISHING JOURNAL OF NEURAL ENGINEERING , 2003 .

[49]  Shirley Coyle,et al.  On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces. , 2004, Physiological measurement.

[50]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[51]  Klaus-Robert Müller,et al.  Introduction to machine learning for brain imaging , 2011, NeuroImage.

[52]  M G Bleichner,et al.  Functional MRI-based identification of brain areas involved in motor imagery for implantable brain–computer interfaces , 2011, Journal of neural engineering.

[53]  Shirley M Coyle,et al.  Brain–computer interface using a simplified functional near-infrared spectroscopy system , 2007, Journal of neural engineering.

[54]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[55]  Bin He,et al.  Differential Electrophysiological Coupling for Positive and Negative BOLD Responses during Unilateral Hand Movements , 2011, The Journal of Neuroscience.

[56]  G. Pfurtscheller,et al.  EEG-based discrimination between imagination of right and left hand movement. , 1997, Electroencephalography and clinical neurophysiology.

[57]  David A. Boas,et al.  Quantification of the cortical contribution to the NIRS signal over the motor cortex using concurrent NIRS-fMRI measurements , 2012, NeuroImage.

[58]  G. Pfurtscheller,et al.  Visualization of sensorimotor areas involved in preparation for hand movement based on classification of μ and central β rhythms in single EEG trials in man , 1994, Neuroscience Letters.

[59]  R. Andersen,et al.  Neural prosthetic control signals from plan activity , 2003, Neuroreport.

[60]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[61]  Boris Reuderink,et al.  Robustness of the Common Spatial Patterns algorithm in the BCI-pipeline , 2008 .

[62]  M J Stokes,et al.  EEG-based communication: a pattern recognition approach. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.