A Haemodynamic Brain–Computer Interface Based on Real-Time Classification of near Infrared Spectroscopy Signals during Motor Imagery and Mental Arithmetic

Over the past decade, an increasing number of studies have investigated near infrared (NIR) spectroscopy for signal acquisition in brain–computer interface (BCI) systems. However, although a BCI relies on classifying brain signals in real-time, the majority of previous studies did not perform real-time NIR spectroscopy signal classification but derived knowledge about the feasibility of NIR spectroscopy for BCI purposes from offline analyses. The present study investigates whether NIR spectroscopy signals evoked by two different mental tasks (i.e. motor imagery and mental arithmetic) can be classified in real-time in order to control a NIR-BCI application. Furthermore, since this is the first study that attempts to distinguish between the haemodynamic responses to these two tasks, we aimed to investigate whether this task-combination is feasible for controlling a NIR-BCI. Twelve healthy participants were asked to control a moving ball on a computer screen by performing motor imagery and mental arithmetic tasks. The real-time classification of their task-specific NIR spectroscopy signals yielded accuracy rates ranging from 45% up to 93%. Offline analyses across all participants showed that both tasks evoked different haemodynamic responses in prefrontal and sensorimotor cortex areas. On the one hand, these results demonstrate the considerable potential of NIR spectroscopy for BCI signal acquisition and the feasibility of the applied mental tasks for NIR-BCI control. On the other hand, since the classification accuracy showed an unsatisfactory stability across measurement sessions, we conclude that further investigations and progress in methodological issues are needed and we discuss further steps that have to be taken until it is conceivable to implement a real-time capable NIR-BCI that works with sufficient accuracy across a large group of individuals.

[1]  M. Tanida,et al.  Relation between asymmetry of prefrontal cortex activities and the autonomic nervous system during a mental arithmetic task: near infrared spectroscopy study , 2004, Neuroscience Letters.

[2]  F. E. Grubbs Procedures for Detecting Outlying Observations in Samples , 1969 .

[3]  T. Braver,et al.  Functional specializations in lateral prefrontal cortex associated with the integration and segregation of information in working memory. , 2006, Cerebral cortex.

[4]  A. Kleinschmidt,et al.  Noninvasive Functional Imaging of Human Brain Using Light , 2000, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[5]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[6]  Cuntai Guan,et al.  Near infrared spectroscopy based brain-computer interface , 2005, International Conference on Experimental Mechanics.

[7]  R. Homan,et al.  Cerebral location of international 10-20 system electrode placement. , 1987, Electroencephalography and clinical neurophysiology.

[8]  Theodore J. Huppert,et al.  Real-time imaging of human brain function by near-infrared spectroscopy using an adaptive general linear model , 2009, NeuroImage.

[9]  M. Jeannerod The representing brain: Neural correlates of motor intention and imagery , 1994, Behavioral and Brain Sciences.

[10]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.

[11]  Christa Neuper,et al.  Single-trial classification of antagonistic oxyhemoglobin responses during mental arithmetic , 2011, Medical & Biological Engineering & Computing.

[12]  Klaus-Robert Müller,et al.  Neurophysiological predictor of SMR-based BCI performance , 2010, NeuroImage.

[13]  M. Tanida,et al.  Relation between mental stress-induced prefrontal cortex activity and skin conditions: A near-infrared spectroscopy study , 2007, Brain Research.

[14]  N. Birbaumer Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[15]  Hasan Ayaz,et al.  Assessment of Cognitive Neural Correlates for a Functional Near Infrared-Based Brain Computer Interface System , 2009, HCI.

[16]  Martin Wolf,et al.  Single-trial classification of motor imagery differing in task complexity: a functional near-infrared spectroscopy study , 2011, Journal of NeuroEngineering and Rehabilitation.

[17]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[18]  A. Villringer,et al.  Cerebral oxygenation changes in response to motor stimulation. , 1996, Journal of applied physiology.

[19]  Hellmuth Obrig,et al.  Separability and cross talk: optimizing dual wavelength combinations for near-infrared spectroscopy of the adult head , 2004, NeuroImage.

[20]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[21]  Christa Neuper,et al.  Focal frontal (de)oxyhemoglobin responses during simple arithmetic. , 2010, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[23]  T. Chau,et al.  Towards a system-paced near-infrared spectroscopy brain–computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state , 2011, Journal of neural engineering.

[24]  Margot J. Taylor,et al.  Is 2+2=4? Meta-analyses of brain areas needed for numbers and calculations , 2011, NeuroImage.

[25]  R. Buxton,et al.  Modeling the hemodynamic response to brain activation , 2004, NeuroImage.

[26]  J. Decety,et al.  Neural mechanisms subserving the perception of human actions , 1999, Trends in Cognitive Sciences.

[27]  Christa Neuper,et al.  Motor imagery and EEG-based control of spelling devices and neuroprostheses. , 2006, Progress in brain research.

[28]  Martin Wolf,et al.  Motor imagery in response to fake feedback measured by functional near-infrared spectroscopy , 2010, NeuroImage.

[29]  Niels Birbaumer,et al.  Hemodynamic brain-computer interfaces for communication and rehabilitation , 2009, Neural Networks.

[30]  G Pfurtscheller,et al.  Current trends in Graz Brain-Computer Interface (BCI) research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[31]  M. Tamura,et al.  Interpretation of near-infrared spectroscopy signals: a study with a newly developed perfused rat brain model. , 2001, Journal of applied physiology.

[32]  Gert Pfurtscheller,et al.  Development, set-up and first results for a one-channel near-infrared spectroscopy system / Entwicklung, Aufbau und vorläufige Ergebnisse eines Einkanal- Nahinfrarot-Spektroskopie-Systems , 2008, Biomedizinische Technik. Biomedical engineering.

[33]  Tom Chau,et al.  Decoding subjective preference from single-trial near-infrared spectroscopy signals , 2009, Journal of neural engineering.

[34]  M. Delazer,et al.  Learning complex arithmetic--an fMRI study. , 2003, Brain research. Cognitive brain research.

[35]  A. Kleinschmidt,et al.  Dissociating neural correlates of cognitive components in mental calculation. , 2001, Cerebral cortex.

[36]  Archana K. Singh,et al.  Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI , 2005, NeuroImage.

[37]  M. Wolf,et al.  Wireless miniaturized in-vivo near infrared imaging. , 2008, Optics express.

[38]  M. Herrmann,et al.  Common brain regions underlying different arithmetic operations as revealed by conjunct fMRI–BOLD activation , 2007, Brain Research.

[39]  Sarah D Power,et al.  Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI , 2012, BMC Research Notes.

[40]  Mamoru Tamura,et al.  Non-synchronous behavior of neuronal activity, oxidative metabolism and blood supply during mental tasks in man , 1994, Neuroscience Letters.

[41]  N. Tzourio-mazoyer,et al.  Neural foundations of logical and mathematical cognition , 2003, Nature Reviews Neuroscience.

[42]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[43]  C. Neuper,et al.  The effect of distinct mental strategies on classification performance for brain-computer interfaces. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[44]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[45]  Ulrich Dirnagl,et al.  Age Dependency of Changes in Cerebral Hemoglobin Oxygenation during Brain Activation: A Near-Infrared Spectroscopy Study , 1995, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[46]  David A. Boas,et al.  A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain Activation , 2002, NeuroImage.

[47]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[48]  A Villringer,et al.  Coupling of brain activity and cerebral blood flow: basis of functional neuroimaging. , 1995, Cerebrovascular and brain metabolism reviews.

[49]  Jian Kong,et al.  The neural substrate of arithmetic operations and procedure complexity. , 2005, Brain research. Cognitive brain research.

[50]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[51]  Kaoru Sakatani,et al.  Effects of fragrance administration on stress-induced prefrontal cortex activity and sebum secretion in the facial skin , 2008, Neuroscience Letters.

[52]  Sarah D Power,et al.  Classification of prefrontal activity due to mental arithmetic and music imagery using hidden Markov models and frequency domain near-infrared spectroscopy , 2010, Journal of neural engineering.

[53]  Kynan Eng,et al.  Trial-to-trial variability differentiates motor imagery during observation between low versus high responders: A functional near-infrared spectroscopy study , 2012, Behavioural Brain Research.

[54]  M. Tamura,et al.  Detection of dynamic changes in cerebral oxygenation coupled to neuronal function during mental work in man , 1993, Neuroscience Letters.

[55]  M. Raichle,et al.  Focal physiological uncoupling of cerebral blood flow and oxidative metabolism during somatosensory stimulation in human subjects. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[56]  C. Markham,et al.  Hemodynamics for Brain-Computer Interfaces , 2008, IEEE Signal Processing Magazine.

[57]  C Neuper,et al.  Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[58]  Masashi Kiguchi,et al.  A Communication Means for Totally Locked-in ALS Patients Based on Changes in Cerebral Blood Volume Measured with Near-Infrared Light , 2007, IEICE Trans. Inf. Syst..

[59]  J. Decety The neurophysiological basis of motor imagery , 1996, Behavioural Brain Research.

[60]  B. Mazoyer,et al.  Neural Correlates of Simple and Complex Mental Calculation , 2001, NeuroImage.

[61]  J. Decety Do imagined and executed actions share the same neural substrate? , 1996, Brain research. Cognitive brain research.

[62]  A. Blasi,et al.  Illuminating the developing brain: The past, present and future of functional near infrared spectroscopy , 2010, Neuroscience & Biobehavioral Reviews.

[63]  T. Chau,et al.  Intersession Consistency of Single-Trial Classification of the Prefrontal Response to Mental Arithmetic and the No-Control State by NIRS , 2012, PloS one.

[64]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.

[65]  Shirley M Coyle,et al.  Brain–computer interface using a simplified functional near-infrared spectroscopy system , 2007, Journal of neural engineering.

[66]  Shirley Coyle,et al.  On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces. , 2004, Physiological measurement.

[67]  Jinung An,et al.  Neural Network Classification of Brain Hemodynamic Responses from Four Mental Tasks , 2011 .

[68]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .