Eyes-closed hybrid brain-computer interface employing frontal brain activation

Brain-computer interfaces (BCIs) have been studied extensively in order to establish a non-muscular communication channel mainly for patients with impaired motor functions. However, many limitations remain for BCIs in clinical use. In this study, we propose a hybrid BCI that is based on only frontal brain areas and can be operated in an eyes-closed state for end users with impaired motor and declining visual functions. In our experiment, electroencephalography (EEG) and near-infrared spectroscopy (NIRS) were simultaneously measured while 12 participants performed mental arithmetic (MA) and remained relaxed (baseline state: BL). To evaluate the feasibility of the hybrid BCI, we classified MA- from BL-related brain activation. We then compared classification accuracies using two unimodal BCIs (EEG and NIRS) and the hybrid BCI in an offline mode. The classification accuracy of the hybrid BCI (83.9 ± 10.3%) was shown to be significantly higher than those of unimodal EEG-based (77.3 ± 15.9%) and NIRS-based BCI (75.9 ± 6.3%). The analytical results confirmed performance improvement with the hybrid BCI, particularly for only frontal brain areas. Our study shows that an eyes-closed hybrid BCI approach based on frontal areas could be applied to neurodegenerative patients who lost their motor functions, including oculomotor functions.

[1]  Keum-Shik Hong,et al.  fNIRS-based brain-computer interfaces: a review , 2015, Front. Hum. Neurosci..

[2]  G. R. Muller,et al.  Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment , 2003, Clinical Neurophysiology.

[3]  Bernhard Schölkopf,et al.  Self-regulation of brain rhythms in the precuneus: a novel BCI paradigm for patients with ALS , 2016, Journal of neural engineering.

[4]  C. Neuper,et al.  Stability of event-related (de-) synchronization during brain–computer interface-relevant mental tasks , 2013, Clinical Neurophysiology.

[5]  Bin Xia,et al.  Brain–Computer Interface–Based Communication in the Completely Locked-In State , 2017, GBCIC.

[6]  Heidrun Wabnitz,et al.  M3BA: A Mobile, Modular, Multimodal Biosignal Acquisition Architecture for Miniaturized EEG-NIRS-Based Hybrid BCI and Monitoring , 2017, IEEE Transactions on Biomedical Engineering.

[7]  Han-Jeong Hwang,et al.  Near-infrared spectroscopy (NIRS)-based eyes-closed brain-computer interface (BCI) using prefrontal cortex activation due to mental arithmetic , 2016, Scientific Reports.

[8]  W. De Clercq,et al.  Automatic Removal of Ocular Artifacts in the EEG without an EOG Reference Channel , 2006, Proceedings of the 7th Nordic Signal Processing Symposium - NORSIG 2006.

[9]  Stefan Haufe,et al.  Multivariate Machine Learning Methods for Fusing Multimodal Functional Neuroimaging Data , 2015, Proceedings of the IEEE.

[10]  Tom Chau,et al.  Towards a ternary NIRS-BCI: single-trial classification of verbal fluency task, Stroop task and unconstrained rest , 2015, Journal of neural engineering.

[11]  Z J Koles,et al.  EEG source localization: implementing the spatio-temporal decomposition approach. , 1998, Electroencephalography and clinical neurophysiology.

[12]  K. Jellinger.  Toward Brain-Computer Interfacing , 2009 .

[13]  A. Kübler,et al.  A Brain–Computer Interface Controlled Auditory Event‐Related Potential (P300) Spelling System for Locked‐In Patients , 2009, Annals of the New York Academy of Sciences.

[14]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[15]  G. Pfurtscheller,et al.  How many people are able to operate an EEG-based brain-computer interface (BCI)? , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[17]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[18]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[19]  Klaus-Robert Müller,et al.  Descriptor : Simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset , 2018 .

[20]  Sarah D Power,et al.  Classification of prefrontal activity due to mental arithmetic and music imagery using hidden Markov models and frequency domain near-infrared spectroscopy , 2010, Journal of neural engineering.

[21]  Chang-Hwan Im,et al.  Corrigendum: Classification of binary intentions for individuals with impaired oculomotor function: 'eyes-closed' SSVEP-based brain-computer interface , 2013 .

[22]  Chang-Hwan Im,et al.  Classification of binary intentions for individuals with impaired oculomotor function: 'eyes-closed' SSVEP-based brain-computer interface (BCI). , 2013, Journal of neural engineering.

[23]  Chang-Hwan Im,et al.  Clinical feasibility of brain-computer interface based on steady-state visual evoked potential in patients with locked-in syndrome: Case studies. , 2017, Psychophysiology.

[24]  Stefan Haufe,et al.  The Berlin Brain-Computer Interface: Progress Beyond Communication and Control , 2016, Front. Neurosci..

[25]  A. Eke,et al.  The modified Beer–Lambert law revisited , 2006, Physics in medicine and biology.

[26]  Christa Neuper,et al.  Motor imagery and EEG-based control of spelling devices and neuroprostheses. , 2006, Progress in brain research.

[27]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[28]  Y. Kim,et al.  Classification of prefrontal and motor cortex signals for three-class fNIRS–BCI , 2015, Neuroscience Letters.

[29]  Klaus-Robert Müller,et al.  Open Access Dataset for EEG+NIRS Single-Trial Classification , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Bernhard Schölkopf,et al.  Closing One's eyes Affects amplitude modulation but not frequency modulation in a Cognitive BCI , 2017, GBCIC.

[31]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[32]  Klaus-Robert Müller,et al.  Machine-Learning-Based Coadaptive Calibration for Brain-Computer Interfaces , 2011, Neural Computation.

[33]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[34]  M. Herrmann,et al.  Prefrontal activation through task requirements of emotional induction measured with NIRS , 2003, Biological Psychology.

[35]  Klaus-Robert Müller,et al.  Motor Imagery for Severely Motor-Impaired Patients: Evidence for Brain-Computer Interfacing as Superior Control Solution , 2014, PloS one.

[36]  Motoaki Kawanabe,et al.  Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing , 2007, NIPS.

[37]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[38]  Andrzej Cichocki,et al.  Bimodal BCI Using Simultaneously NIRS and EEG , 2014, IEEE Transactions on Biomedical Engineering.

[39]  Klaus-Robert Müller,et al.  Co-adaptive calibration to improve BCI efficiency , 2011, Journal of neural engineering.

[40]  L. Cohen,et al.  Brain–machine interface in chronic stroke rehabilitation: A controlled study , 2013, Annals of neurology.

[41]  Klaus-Robert Müller,et al.  Learning From More Than One Data Source: Data Fusion Techniques for Sensorimotor Rhythm-Based Brain–Computer Interfaces , 2015, Proceedings of the IEEE.

[42]  Stefan Haufe,et al.  The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology , 2010, Front. Neurosci..

[43]  Tanja Schultz,et al.  Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS , 2014, Front. Hum. Neurosci..

[44]  J. Wolpaw,et al.  Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface , 2005, Neurology.

[45]  Gerwin Schalk,et al.  Brain–computer interfacing based on cognitive control , 2010, Annals of neurology.

[46]  T. Chau,et al.  Towards a system-paced near-infrared spectroscopy brain–computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state , 2011, Journal of neural engineering.

[47]  Bernhard Schölkopf,et al.  A Cognitive Brain-Computer Interface for Patients with Amyotrophic Lateral Sclerosis , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[48]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[49]  T. Chau,et al.  Automatic detection of a prefrontal cortical response to emotionally rated music using multi-channel near-infrared spectroscopy , 2012, Journal of neural engineering.

[50]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[51]  D. Ulrich,et al.  A Gaze Independent Brain-Computer Interface Based on Visual Stimulation through Closed Eyelids , 2015, Scientific Reports.

[52]  N. Birbaumer.  Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[53]  W. Freeman,et al.  Spatial spectra of scalp EEG and EMG from awake humans , 2003, Clinical Neurophysiology.

[54]  Mark S. Cohen,et al.  Simultaneous EEG and fMRI of the alpha rhythm , 2002, Neuroreport.

[55]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[56]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[57]  Arie Yeredor,et al.  A fast approximate joint diagonalization algorithm using a criterion with a block diagonal weight matrix , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.