Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy

Passive brain-computer interfaces consider brain activity as an additional source of information, to augment and adapt the interface instead of controlling it. We have developed a software system that allows for real time brain signal analysis and machine learning classification of affective and workload states measured with functional near-infrared spectroscopy (fNIRS) called the online fNIRS analysis and classification (OFAC). Our system reproduces successful offline procedures, adapting them for real-time input to a user interface. Our first evaluation compares a previous offline analysis with our online analysis. While results show an accuracy decrease, they are outweighed by the new ability of interface adaptation. The second study demonstrates OFAC's online features through real-time classification of two tasks, and interface adaptation according to the predicted task. Accuracy averaged over 85%. We have created the first working real time passive BCI using fNIRS, opening the door to build adaptive user interfaces.

[1]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[2]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.

[3]  Shirley M Coyle,et al.  Brain–computer interface using a simplified functional near-infrared spectroscopy system , 2007, Journal of neural engineering.

[4]  Ellen S. Wilschut,et al.  Brain–machine interfaces in space: Using spontaneous rather than intentionally generated brain signals , 2010 .

[5]  Kari Kallinen,et al.  The effects of background music on using a pocket computer in a cafeteria: immersion, emotional responses, and social richness of medium , 2004, CHI EA '04.

[6]  N. Mukawa,et al.  Brain Activity Comparison of Different-Genre Video Game Players , 2007, Second International Conference on Innovative Computing, Informatio and Control (ICICIC 2007).

[7]  Junichiro Wakatsuki,et al.  Changes in Concentration of Oxygenated Hemoglobin in the Prefrontal Cortex while Absorbed in Listening to Music , 2009, 2009 International Conference on Biometrics and Kansei Engineering.

[8]  J. Leon-Carrion,et al.  A lasting post-stimulus activation on dorsolateral prefrontal cortex is produced when processing valence and arousal in visual affective stimuli , 2007, Neuroscience Letters.

[9]  Robert Oostenveld,et al.  MATLAB-Based Tools for BCI Research , 2010, Brain-Computer Interfaces.

[10]  Thomas G. Dietterich.  Machine Learning for Sequential Data: A Review , 2002, SSPR/SPR.

[11]  B. Chance,et al.  A novel method for fast imaging of brain function, non-invasively, with light. , 1998, Optics express.

[12]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[13]  Klaus-Robert Müller,et al.  Berlin Brain-Computer Interface - The HCI communication channel for discovery , 2007, Int. J. Hum. Comput. Stud..

[14]  Tom Chau,et al.  Decoding subjective preference from single-trial near-infrared spectroscopy signals , 2009, Journal of neural engineering.

[15]  M. Leanne,et al.  Human-Computer Interaction and Brain Measurement Using Functional Near-Infrared Spectroscopy , 2007 .

[16]  S. Bunce,et al.  Functional near-infrared spectroscopy , 2006, IEEE Engineering in Medicine and Biology Magazine.

[17]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines , 2009, UIST '09.

[18]  Desney S. Tan,et al.  Using a low-cost electroencephalograph for task classification in HCI research , 2006, UIST.

[19]  Clemens Brunner,et al.  BioSig: A Free and Open Source Software Library for BCI Research , 2008, Computer.

[20]  Wen-Hung Huang,et al.  Effects of music tempo and task difficulty on multi-attribute decision-making: An eye-tracking approach , 2009, Comput. Hum. Behav..

[21]  Robert J. K. Jacob,et al.  Distinguishing Difficulty Levels with Non-invasive Brain Activity Measurements , 2009, INTERACT.

[22]  Thomas E. Nichols,et al.  Activation of the medial prefrontal cortex and extended amygdala by individual ratings of emotional arousal: a fMRI study , 2003, Biological Psychiatry.

[23]  Edward Cutrell,et al.  BCI for passive input in HCI , 2007 .

[24]  Melody Moore Jackson,et al.  An fNIR based BMI for letter construction using continuous control , 2009, CHI Extended Abstracts.

[25]  Thorsten O. Zander,et al.  Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces , 2010, Brain-Computer Interfaces.

[26]  José del R. Millán,et al.  BioSig: An Open-Source Software Library for BCI Research , 2007 .

[27]  Britton Chance,et al.  Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks , 2004, Int. J. Hum. Comput. Interact..

[28]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[29]  Theodore J. Huppert,et al.  Real-time imaging of human brain function by near-infrared spectroscopy using an adaptive general linear model , 2009, NeuroImage.