An Evaluation of Training with an Auditory P300 Brain-Computer Interface for the Japanese Hiragana Syllabary

Gaze-independent brain-computer interfaces (BCIs) are a possible communication channel for persons with paralysis. We investigated if it is possible to use auditory stimuli to create a BCI for the Japanese Hiragana syllabary, which has 46 Hiragana characters. Additionally, we investigated if training has an effect on accuracy despite the high amount of different stimuli involved. Able-bodied participants (N = 6) were asked to select 25 syllables (out of fifty possible choices) using a two step procedure: First the consonant (ten choices) and then the vowel (five choices). This was repeated on 3 separate days. Additionally, a person with spinal cord injury (SCI) participated in the experiment. Four out of six healthy participants reached Hiragana syllable accuracies above 70% and the information transfer rate increased from 1.7 bits/min in the first session to 3.2 bits/min in the third session. The accuracy of the participant with SCI increased from 12% (0.2 bits/min) to 56% (2 bits/min) in session three. Reliable selections from a 10 × 5 matrix using auditory stimuli were possible and performance is increased by training. We were able to show that auditory P300 BCIs can be used for communication with up to fifty symbols. This enables the use of the technology of auditory P300 BCIs with a variety of applications.

[1]  J. Wolpaw,et al.  A practical, intuitive brain–computer interface for communicating ‘yes’ or ‘no’ by listening , 2014, Journal of neural engineering.

[2]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[3]  G. McCarthy,et al.  Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time. , 1977, Science.

[4]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[5]  D. Ulrich,et al.  A Gaze Independent Brain-Computer Interface Based on Visual Stimulation through Closed Eyelids , 2015, Scientific Reports.

[6]  Chang S. Nam,et al.  Environmental Noise and P300-Based Brain-Computer Interface (BCI) , 2008 .

[7]  C. Neuper,et al.  Toward a high-throughput auditory P300-based brain–computer interface , 2009, Clinical Neurophysiology.

[8]  G Müller-Putz,et al.  An independent SSVEP-based brain–computer interface in locked-in syndrome , 2014, Journal of neural engineering.

[9]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[10]  Ivo Käthner,et al.  An auditory multiclass brain-computer interface with natural stimuli: Usability evaluation with healthy participants and a motor impaired end user , 2015, Front. Hum. Neurosci..

[11]  Toshio Shimizu,et al.  ALS patients with ability to communicate after long-term mechanical ventilation have confined degeneration to the motor neuron system , 2016, Journal of the Neurological Sciences.

[12]  N. Birbaumer,et al.  An auditory oddball brain–computer interface for binary choices , 2010, Clinical Neurophysiology.

[13]  B. Blankertz,et al.  A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue , 2010, PloS one.

[14]  Tobias Kaufmann,et al.  Comparison of tactile, auditory, and visual modality for brain-computer interface use: a case study with a patient in the locked-in state , 2013, Front. Neurosci..

[15]  Dean J Krusienski,et al.  A comparison of classification techniques for the P300 Speller , 2006, Journal of neural engineering.

[16]  E. Donchin,et al.  On quantifying surprise: the variation of event-related potentials with subjective probability. , 1977, Psychophysiology.

[17]  Andrea Kübler,et al.  Empathy, motivation, and P300 BCI performance , 2013, Front. Hum. Neurosci..

[18]  Tomasz M. Rutkowski,et al.  Tactile and bone-conduction auditory brain computer interface for vision and hearing impaired users , 2014, Journal of Neuroscience Methods.

[19]  B. Schoelkopf,et al.  Transition from the locked in to the completely locked-in state: A physiological analysis , 2011, Clinical Neurophysiology.

[20]  A. Kübler,et al.  Motivation modulates the P300 amplitude during brain–computer interface use , 2010, Clinical Neurophysiology.

[21]  Jan B. F. van Erp,et al.  A Tactile P300 Brain-Computer Interface , 2010, Front. Neurosci..

[22]  Bernhard Schölkopf,et al.  An Auditory Paradigm for Brain-Computer Interfaces , 2004, NIPS.

[23]  Konstantinos Priftis,et al.  Effectiveness of the P3-speller in brain–computer interfaces for amyotrophic lateral sclerosis patients: a systematic review and meta-analysis , 2014, Front. Neuroeng..

[24]  Tobias Kaufmann,et al.  The WIN-speller: a new intuitive auditory brain-computer interface spelling application , 2015, Front. Neurosci..

[25]  Wolfgang Rosenstiel,et al.  Prediction of P300 BCI Aptitude in Severe Motor Impairment , 2013, PloS one.

[26]  N Birbaumer,et al.  Cognitive processing in completely paralyzed patients with amyotrophic lateral sclerosis , 2003, European journal of neurology.

[27]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[28]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[29]  T. Takasu,et al.  Amyotrophic lateral sclerosis with ophthalmoplegia and multisystem degeneration in patients on long-term use of respirators , 2004, Acta Neuropathologica.

[30]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[31]  D. Zee,et al.  The neuro-ophthalmology of multiple sclerosis , 2005, The Lancet Neurology.

[32]  Shuichi Kato,et al.  Total manifestations of amyotrophic lateral sclerosis ALS in the totally locked-in state , 1989, Journal of the Neurological Sciences.

[33]  A. Kübler,et al.  Effects of training and motivation on auditory P300 brain–computer interface performance , 2016, Clinical Neurophysiology.

[34]  Klaus-Robert Müller,et al.  Neurophysiological predictor of SMR-based BCI performance , 2010, NeuroImage.

[35]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[36]  Wolfgang Rosenstiel,et al.  Neural mechanisms of brain–computer interface control , 2011, NeuroImage.

[37]  Melinda C. Anderson,et al.  Chosen Listening Levels for Music With and Without the Use of Hearing Aids. , 2016, American journal of audiology.

[38]  Toshio Shimizu,et al.  Marked preservation of the visual and olfactory pathways in ALS patients in a totally locked-in state. , 2015, Clinical neuropathology.

[39]  David J. Mack,et al.  Video game players show higher performance but no difference in speed of attention shifts. , 2016, Acta psychologica.

[40]  Shoji Makino,et al.  Spatial Auditory Two-step Input Japanese Syllabary Brain-computer Interface Speller , 2014 .

[41]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[42]  F. Cincotti,et al.  Eye-gaze independent EEG-based brain–computer interfaces for communication , 2012, Journal of neural engineering.

[43]  Takeshi Sakurada,et al.  A BMI-based occupational therapy assist suit: asynchronous control by SSVEP , 2013, Front. Neurosci..

[44]  Christoph Braun,et al.  A portable auditory P300 brain–computer interface with directional cues , 2013, Clinical Neurophysiology.

[45]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[46]  Chang-Hwan Im,et al.  Classification of binary intentions for individuals with impaired oculomotor function: 'eyes-closed' SSVEP-based brain-computer interface (BCI). , 2013, Journal of neural engineering.

[47]  Stefano Federici,et al.  Usability and Workload of Access Technology for People With Severe Motor Impairment , 2015, Neurorehabilitation and neural repair.

[48]  Kouji Takano,et al.  A region-based two-step P300-based brain–computer interface for patients with amyotrophic lateral sclerosis , 2014, Clinical Neurophysiology.

[49]  Minoru Kamata,et al.  The development of a brain computer interface device for amyotrophic lateral sclerosis patients , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[50]  A. Kübler,et al.  Training leads to increased auditory brain–computer interface performance of end-users with motor impairments , 2016, Clinical Neurophysiology.

[51]  Toshihiro Kawase,et al.  Use of high-frequency visual stimuli above the critical flicker frequency in a SSVEP-based BMI , 2015, Clinical Neurophysiology.