Sensorimotor learning with stereo auditory feedback for a brain–computer interface

Motor imagery can be used to modulate sensorimotor rhythms (SMR) enabling detection of voltage fluctuations on the surface of the scalp using electroencephalographic electrodes. Feedback is essential in learning to modulate SMR for non-muscular communication using a brain–computer interface (BCI). A BCI not reliant upon the visual modality not only releases the visual channel for other uses but also offers an attractive means of communication for the physically impaired who are also blind or vision impaired. This study demonstrates the feasibility of replacing the traditional visual feedback modality with stereo auditory feedback. Results from a pilot study were used to select the most appropriate sounds for auditory feedback based on three options: broadband noise and two anechoic instrument samples. Subsequently, an SMR BCI was used to examine the effect on sensorimotor learning with broadband noise utilising a modified stereophonic presentation method. Twenty participants split into equal groups took part in ten sessions. The visual group performed best initially but did not improve over time whilst the auditory group improved as the study progressed. The results demonstrate the feasibility of using stereophonic auditory feedback with broadband noise as opposed to other auditory feedback presentation methods and sounds which are less intuitive.

[1]  Benjamin Blankertz,et al.  A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System , 2011, Front. Neurosci..

[2]  M. Ohki,et al.  Ocular abnormalities in amyotrophic lateral sclerosis. , 1994, Acta oto-laryngologica. Supplementum.

[3]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[4]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[5]  Klaus-Robert Müller,et al.  On Optimal Channel Configurations for SMR-based Brain–Computer Interfaces , 2010, Brain Topography.

[6]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[7]  N. Birbaumer,et al.  An auditory oddball brain–computer interface for binary choices , 2010, Clinical Neurophysiology.

[8]  H. Flor,et al.  A multimodal brain-based feedback and communication system , 2004, Experimental Brain Research.

[9]  Damien Coyle,et al.  Achievable Brain Computer Communication through Short Intensive Motor Imagery Training despite Long Term Spinal Cord Injury , 2011 .

[10]  Damien Coyle,et al.  Operating a Brain Computer Interface: Able Bodied vs. Physically Impaired Performance , 2011 .

[11]  Benjamin Blankertz,et al.  Two-dimensional auditory p300 speller with predictive text system , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[12]  R. Burns BLUMLEIN AND THE BIRTH OF STEREO , 1999 .

[13]  W. Hartmann Localization of sound in rooms. , 1983, The Journal of the Acoustical Society of America.

[14]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .

[15]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[16]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[17]  Francisco Velasco-Álvarez,et al.  Audio-Cued SMR Brain-Computer Interface to Drive a Virtual Wheelchair , 2011, IWANN.

[18]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[19]  J. Blauert Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[20]  Ville Pulkki,et al.  Compensating Displacement of Amplitude-panned Virtual Sources , 2002 .

[21]  Thilo Hinterberger,et al.  An Auditory Brain-Computer Interface Based on the Self-Regulation of Slow Cortical Potentials , 2005, Neurorehabilitation and neural repair.

[22]  Thilo Hinterberger,et al.  POSER: Parametric Orchestral Sonification of EEG in Real-Time for the Self-Regulation of Brain States , 2004 .

[23]  Christa Neuper,et al.  Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb , 2011, Medical & Biological Engineering & Computing.

[24]  Jun Lv,et al.  Common Spatial Pattern and Particle Swarm Optimization for Channel Selection in BCI , 2008, 2008 3rd International Conference on Innovative Computing Information and Control.

[25]  A Kübler,et al.  A P 300-based brain-computer interface for people with amyotrophic lateral sclerosis , 2010 .

[26]  Christa Neuper,et al.  Impact of auditory distraction on user performance in a brain–computer interface driven by different mental tasks , 2011, Clinical Neurophysiology.

[27]  Thilo Hinterberger,et al.  ORCHESTRAL SONIFICATION OF BRAIN SIGNALS AND ITS APPLICATION TO BRAIN-COMPUTER INTERFACES AND PERFORMING ARTS , 2007 .

[28]  T. Martin McGinnity,et al.  EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game , 2011, 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[29]  G. Pfurtscheller,et al.  Rapid prototyping of an EEG-based brain-computer interface (BCI) , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  N. Birbaumer,et al.  The Influence of Psychological State and Motivation on Brain–Computer Interface Performance in Patients with Amyotrophic Lateral Sclerosis – a Longitudinal Study , 2010, Front. Neuropharma..

[31]  Douglas S Brungart,et al.  Auditory localization in the horizontal plane with single and double hearing protection. , 2002, Aviation, space, and environmental medicine.

[32]  Thilo Hinterberger,et al.  Parametric orchestral sonification of EEG in real time , 2005, IEEE MultiMedia.

[33]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[34]  T.M. McGinnity,et al.  A time-series prediction approach for feature extraction in a brain-computer interface , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  B. Blankertz,et al.  A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue , 2010, PloS one.

[36]  Andrzej Cichocki,et al.  Spatial Auditory BCI/BMI Paradigm - Multichannel EMD Approach to Brain Responses Estimation , 2010 .

[37]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[38]  Girijesh Prasad,et al.  A Covariate Shift Minimisation Method to Alleviate Non-stationarity Effects for an Adaptive Brain-Computer Interface , 2010, 2010 20th International Conference on Pattern Recognition.