Is Sensorimotor BCI Performance Influenced Differently by Mono, Stereo, or 3-D Auditory Feedback?

Imagination of movement can be used as a control method for a brain-computer interface (BCI) allowing communication for the physically impaired. Visual feedback within such a closed loop system excludes those with visual problems and hence there is a need for alternative sensory feedback pathways. In the context of substituting the visual channel for the auditory channel, this study aims to add to the limited evidence that it is possible to substitute visual feedback for its auditory equivalent and assess the impact this has on BCI performance. Secondly, the study aims to determine for the first time if the type of auditory feedback method influences motor imagery performance significantly. Auditory feedback is presented using a stepped approach of single (mono), double (stereo), and multiple (vector base amplitude panning as an audio game) loudspeaker arrangements. Visual feedback involves a ball-basket paradigm and a spaceship game. Each session consists of either auditory or visual feedback only with runs of each type of feedback presentation method applied in each session. Results from seven subjects across five sessions of each feedback type (visual, auditory) (10 sessions in total) show that auditory feedback is a suitable substitute for the visual equivalent and that there are no statistical differences in the type of auditory feedback presented across five sessions.

[1]  Chang-Hwan Im,et al.  Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing , 2011, Journal of Neuroscience Methods.

[2]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[3]  Thilo Hinterberger,et al.  Parametric orchestral sonification of EEG in real time , 2005, IEEE MultiMedia.

[4]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[5]  H. Flor,et al.  A multimodal brain-based feedback and communication system , 2004, Experimental Brain Research.

[6]  T.M. McGinnity,et al.  A time-series prediction approach for feature extraction in a brain-computer interface , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  A. Kübler,et al.  A Brain–Computer Interface Controlled Auditory Event‐Related Potential (P300) Spelling System for Locked‐In Patients , 2009, Annals of the New York Academy of Sciences.

[8]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[9]  Girijesh Prasad,et al.  Sensorimotor-rhythm modulation feedback with 3D vector-base amplitude panning — A brain-computer interface pilot study , 2012 .

[10]  Ville Pulkki,et al.  Virtual Sound Source Positioning Using Vector Base Amplitude Panning , 1997 .

[11]  Douglas S Brungart,et al.  Auditory localization in the horizontal plane with single and double hearing protection. , 2002, Aviation, space, and environmental medicine.

[12]  Steven Laureys,et al.  AM: The vegetative state , 2022 .

[13]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[14]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[15]  Matthew H. Davis,et al.  Detecting Awareness in the Vegetative State , 2006, Science.

[16]  Klaus-Robert Müller,et al.  The Berlin Brain-Computer Interface: Accurate performance from first-session in BCI-naive subjects , 2008, IEEE Transactions on Biomedical Engineering.

[17]  Thilo Hinterberger,et al.  ORCHESTRAL SONIFICATION OF BRAIN SIGNALS AND ITS APPLICATION TO BRAIN-COMPUTER INTERFACES AND PERFORMING ARTS , 2007 .

[18]  T. Martin McGinnity,et al.  EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game , 2011, 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[19]  Benjamin Blankertz,et al.  A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System , 2011, Front. Neurosci..

[20]  M. Ohki,et al.  Ocular abnormalities in amyotrophic lateral sclerosis. , 1994, Acta oto-laryngologica. Supplementum.

[21]  Damien Coyle,et al.  Enabling Control in the Minimally Conscious State in a Single Session with a Three Channel BCI , 2012 .

[22]  Nitish V. Thakor.  IEEE Transactions on Neural Systems and Rehabilitation Engineering: Editorial , 2006 .

[23]  Dario Farina,et al.  Offline Identification of Imagined Speed of Wrist Movements in Paralyzed ALS Patients from Single-Trial EEG , 2009, Front. Neuropro..

[24]  Damien Coyle,et al.  Achievable Brain Computer Communication through Short Intensive Motor Imagery Training despite Long Term Spinal Cord Injury , 2011 .

[25]  Thilo Hinterberger,et al.  POSER: Parametric Orchestral Sonification of EEG in Real-Time for the Self-Regulation of Brain States , 2004 .

[26]  R. Burns.  BLUMLEIN AND THE BIRTH OF STEREO , 1999 .

[27]  Damien Coyle,et al.  Visual and Stereo Audio Sensorimotor Rhythm Feedback in the Minimally Conscious State , 2013 .

[28]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[29]  W. Hartmann.  Localization of sound in rooms. , 1983, The Journal of the Acoustical Society of America.

[30]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[31]  B. Blankertz,et al.  A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue , 2010, PloS one.

[32]  E. Sellers,et al.  How many people are able to control a P300-based brain–computer interface (BCI)? , 2009, Neuroscience Letters.

[33]  Andrzej Cichocki,et al.  Spatial Auditory BCI/BMI Paradigm - Multichannel EMD Approach to Brain Responses Estimation , 2010 .

[34]  Girijesh Prasad,et al.  Learning to modulate sensorimotor rhythms with stereo auditory feedback for a brain-computer interface , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[36]  Damien Coyle,et al.  Neural network based auto association and time-series prediction for biosignal processing in brain-computer interfaces , 2009, IEEE Computational Intelligence Magazine.

[37]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[38]  N. Birbaumer,et al.  An auditory oddball brain–computer interface for binary choices , 2010, Clinical Neurophysiology.

[39]  Francisco Velasco-Álvarez,et al.  Audio-Cued SMR Brain-Computer Interface to Drive a Virtual Wheelchair , 2011, IWANN.

[40]  Thilo Hinterberger,et al.  An Auditory Brain-Computer Interface Based on the Self-Regulation of Slow Cortical Potentials , 2005, Neurorehabilitation and neural repair.

[41]  Manuel Rodriguez,et al.  Mu rhythm, visual processing and motor control , 2012, Clinical Neurophysiology.

[42]  Srivas Chennu,et al.  Bedside detection of awareness in the vegetative state: a cohort study , 2011, The Lancet.

[43]  Girijesh Prasad,et al.  Sensorimotor learning with stereo auditory feedback for a brain–computer interface , 2012, Medical & Biological Engineering & Computing.

[44]  Shinsuke Shimojo,et al.  Distortions of Subjective Time Perception Within and Across Senses , 2008, PloS one.