Classification of Overt and Covert Speech for Near-Infrared Spectroscopy-Based Brain Computer Interface

People suffering from neuromuscular disorders such as locked-in syndrome (LIS) are left in a paralyzed state with preserved awareness and cognition. In this study, it was hypothesized that changes in local hemodynamic activity, due to the activation of Broca’s area during overt/covert speech, can be harnessed to create an intuitive Brain Computer Interface based on Near-Infrared Spectroscopy (NIRS). A 12-channel square template was used to cover inferior frontal gyrus and changes in hemoglobin concentration corresponding to six aloud (overtly) and six silently (covertly) spoken words were collected from eight healthy participants. An unsupervised feature extraction algorithm was implemented with an optimized support vector machine for classification. For all participants, when considering overt and covert classes regardless of words, classification accuracy of 92.88 ± 18.49% was achieved with oxy-hemoglobin (O2Hb) and 95.14 ± 5.39% with deoxy-hemoglobin (HHb) as a chromophore. For a six-active-class problem of overtly spoken words, 88.19 ± 7.12% accuracy was achieved for O2Hb and 78.82 ± 15.76% for HHb. Similarly, for a six-active-class classification of covertly spoken words, 79.17 ± 14.30% accuracy was achieved with O2Hb and 86.81 ± 9.90% with HHb as an absorber. These results indicate that a control paradigm based on covert speech can be reliably implemented into future Brain–Computer Interfaces (BCIs) based on NIRS.

[1]  Atsushi Maki,et al.  Non-invasive assessment of language dominance with near-infrared spectroscopic mapping , 1998, Neuroscience Letters.

[2]  Koichi Mori,et al.  Assessing cerebral representations of short and long vowel categories by NIRS , 2002, Neuroreport.

[3]  K. Sakatani,et al.  Language-activated cerebral blood oxygenation and hemodynamic changes of the left prefrontal cortex in poststroke aphasic patients: a near-infrared spectroscopy study. , 1998, Stroke.

[4]  A. Mazzucchi,et al.  Locked-in syndrome: improvement in the prognosis after an early intensive multidisciplinary rehabilitation. , 2003, Archives of physical medicine and rehabilitation.

[5]  A. Villringer,et al.  Decrease in parietal cerebral hemoglobin oxygenation during performance of a verbal fluency task in patients with Alzheimer's disease monitored by means of near-infrared spectroscopy (NIRS) — correlation with simultaneous rCBF-PET measurements , 1997, Brain Research.

[6]  H. Bortfeld,et al.  Near-Infrared Spectroscopy and Cortical Responses to Speech Production , 2009, The Open Neuroimaging Journal.

[7]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[8]  M. García-Fiñana,et al.  Sulcal variability, stereological measurement and asymmetry of Broca's area on MR images , 2007, Journal of anatomy.

[9]  Yoko Hoshi,et al.  Near-Infrared Optical Detection of Sequential Brain Activation in the Prefrontal Cortex during Mental Tasks , 1997, NeuroImage.

[10]  D. Boas,et al.  Non-invasive neuroimaging using near-infrared light , 2002, Biological Psychiatry.

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  Alex Mihailidis,et al.  Assessing the potential of electrodermal activity as an alternative access pathway. , 2008, Medical engineering & physics.

[13]  Christian Laugier,et al.  Controlling a Wheelchair Indoors Using Thought , 2007, IEEE Intelligent Systems.

[14]  Masashi Kiguchi,et al.  A Communication Means for Totally Locked-in ALS Patients Based on Changes in Cerebral Blood Volume Measured with Near-Infrared Light , 2007, IEICE Trans. Inf. Syst..

[15]  Jeremy I. Skipper,et al.  Speech-associated gestures, Broca’s area, and the human mirror system , 2007, Brain and Language.

[16]  C. A. Ruf,et al.  Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy , 2014, Neurology.

[17]  C. Giussani,et al.  Review of language organisation in bilingual patients: what can we learn from direct brain mapping? , 2007, Acta Neurochirurgica.

[18]  Gernot R. Müller-Putz,et al.  "Virtual keyboard" controlled by spontaneous EEG activity , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Arthur W. Toga,et al.  Functional assessment of broca's area using near infrared spectroscopy in humans , 2001, NeuroImage.

[20]  A. Ehlis,et al.  Influence of muscle activity on brain oxygenation during verbal fluency assessed with functional near-infrared spectroscopy , 2010, Neuroscience.

[21]  B. Schoelkopf,et al.  Transition from the locked in to the completely locked-in state: A physiological analysis , 2011, Clinical Neurophysiology.

[22]  J. Selhorst,et al.  "Locked-in" syndrome. , 1987, Stroke.

[23]  E. Watanabe,et al.  Non-invasive functional mapping with multi-channel near infra-red spectroscopic topography in humans , 1996, Neuroscience Letters.

[24]  Marco Ferrari,et al.  Lateral frontal cortex oxygenation changes during translation and language switching revealed by non-invasive near-infrared multi-point measurements , 2002, Brain Research Bulletin.

[25]  Steven Laureys,et al.  The locked-in syndrome : what is it like to be conscious but paralyzed and voiceless? , 2005, Progress in brain research.

[26]  Yuanqing Li,et al.  Surfing the internet with a BCI mouse , 2012, Journal of neural engineering.

[27]  A. Fallgatter,et al.  Prefrontal Hypooxygenation during Language Processing Assessed with Near-Infrared Spectroscopy , 1998, Neuropsychobiology.

[28]  Steven Laureys,et al.  Locked-in syndrome in children: report of five cases and review of the literature. , 2009, Pediatric neurology.

[29]  R. Homan,et al.  Cerebral location of international 10-20 system electrode placement. , 1987, Electroencephalography and clinical neurophysiology.

[30]  Ann-Christine Ehlis,et al.  Optical topography during a Go–NoGo task assessed with multi-channel near-infrared spectroscopy , 2005, Behavioural Brain Research.

[31]  Martin Wolf,et al.  Single-trial classification of motor imagery differing in task complexity: a functional near-infrared spectroscopy study , 2011, Journal of NeuroEngineering and Rehabilitation.

[32]  Linda Ko,et al.  Near-infrared Spectroscopy as an Access Channel: Prefrontal Cortex Inhibition During an Auditory Go-no-go Task , 2009 .

[33]  Sun Qiliang Language-activated cerebral blood oxygenation and hemodynamic changes of the left prefrontal cortex in poststroke aphasic patients:A near infrared spectroscopy study , 2004 .

[34]  K. Sakai,et al.  Temporal cortex activation during speech recognition: an optical topography study , 1999, Cognition.

[35]  H. Flor,et al.  Non-invasive functional mapping of the human motor cortex using near-infrared spectroscopy. , 1996, Neuroreport.

[36]  A. Villringer,et al.  Noninvasive Assessment of Changes in Cytochrome-c Oxidase Oxidation in Human Subjects during Visual Stimulation , 1999, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[37]  A. Braun,et al.  Activation of Broca’s area during the production of spoken and signed language: a combined cytoarchitectonic mapping and PET analysis , 2003, Neuropsychologia.

[38]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.