Eye-gaze independent EEG-based brain–computer interfaces for communication

The present review systematically examines the literature reporting gaze independent interaction modalities in non-invasive brain-computer interfaces (BCIs) for communication. BCIs measure signals related to specific brain activity and translate them into device control signals. This technology can be used to provide users with severe motor disability (e.g. late stage amyotrophic lateral sclerosis (ALS); acquired brain injury) with an assistive device that does not rely on muscular contraction. Most of the studies on BCIs explored mental tasks and paradigms using visual modality. Considering that in ALS patients the oculomotor control can deteriorate and also other potential users could have impaired visual function, tactile and auditory modalities have been investigated over the past years to seek alternative BCI systems which are independent from vision. In addition, various attentional mechanisms, such as covert attention and feature-directed attention, have been investigated to develop gaze independent visual-based BCI paradigms. Three areas of research were considered in the present review: (i) auditory BCIs, (ii) tactile BCIs and (iii) independent visual BCIs. Out of a total of 130 search results, 34 articles were selected on the basis of pre-defined exclusion criteria. Thirteen articles dealt with independent visual BCIs, 15 reported on auditory BCIs and the last six on tactile BCIs, respectively. From the review of the available literature, it can be concluded that a crucial point is represented by the trade-off between BCI systems/paradigms with high accuracy and speed, but highly demanding in terms of attention and memory load, and systems requiring lower cognitive effort but with a limited amount of communicable information. These issues should be considered as priorities to be explored in future studies to meet users' requirements in a real-life scenario.

[1]  H. L. Andrews,et al.  ELECTRO-ENCEPHALOGRAPHY: III. NORMAL DIFFERENTIATION OF OCCIPITAL AND PRECENTRAL REGIONS IN MAN , 1938 .

[2]  Electro-encephalography. III: Normal Differentiation of Occipital and Precentral Regions in Man. (Arch. Neur. and Psychiat., vol. xxxix, p. 96, Jan., 1938.) Jasper, H. H., and Andrews, H. L. , 1938 .

[3]  H. Jasper,et al.  Electrocorticograms in man: Effect of voluntary movement upon the electrical activity of the precentral gyrus , 1949 .

[4]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[5]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[6]  R J Sclabassi,et al.  Somatosensory responses to stimulus trains: normative data. , 1974, Electroencephalography and clinical neurophysiology.

[7]  G. Pfurtscheller,et al.  Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement. , 1979, Electroencephalography and clinical neurophysiology.

[8]  B. Rockstroh,et al.  Biofeedback of slow cortical potentials. I. , 1980, Electroencephalography and clinical neurophysiology.

[9]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[10]  Emanuel Donchin,et al.  Definition, Identification, and Reliability of Measurement of the P300 Component of the Event-Related Brain Potential , 1987 .

[11]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[12]  Erich E. Sutter,et al.  The brain response interface: communication through visually-induced electrical brain responses , 1992 .

[13]  G. Pfurtscheller,et al.  Simultaneous EEG 10 Hz desynchronization and 40 Hz synchronization during finger movements. , 1992, Neuroreport.

[14]  H. J. Eysenck,et al.  Advances in psychophysiology: J.R. Jennings, P.K. Ackles & M.G.H. Coles (Eds) Vol.5 (1993).320 pp. £42.50 (hardback). ISBN 185302 191 1 , 1994 .

[15]  P. Massman,et al.  Prevalence and correlates of neuropsychological deficits in amyotrophic lateral sclerosis. , 1996, Journal of neurology, neurosurgery, and psychiatry.

[16]  T. Münte,et al.  Relation of neuropsychological and magnetic resonance findings in amyotrophic lateral sclerosis: evidence for subgroups , 1997, Clinical Neurology and Neurosurgery.

[17]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[18]  D J McFarland,et al.  Brain-computer interface research at the Wadsworth Center. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[19]  H. Flor,et al.  The thought translation device (TTD) for completely paralyzed patients. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[20]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[21]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[22]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[23]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[24]  Xiaorong Gao,et al.  Design and implementation of a brain-computer interface with high transfer rates , 2002, IEEE Transactions on Biomedical Engineering.

[25]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[26]  Xiaorong Gao,et al.  A BCI-based environmental controller for the motion-disabled. , 2003, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[27]  N. Birbaumer,et al.  Conscious perception of brain states: mental strategies for brain–computer communication , 2003, Neuropsychologia.

[28]  B. Scholkopf,et al.  Attention modulation of auditory event-related potentials in a brain-computer interface , 2004, IEEE International Workshop on Biomedical Circuits and Systems, 2004..

[29]  E. Lalor,et al.  A comparison of covert and overt attention as a control option in a steady-state visual evoked potential-based brain computer interface , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[30]  R. Veit,et al.  Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI) , 2004, Journal of Physiology-Paris.

[31]  Shirley Coyle,et al.  On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces. , 2004, Physiological measurement.

[32]  Nikolaus Weiskopf,et al.  An EEG-driven brain-computer interface combined with functional magnetic resonance imaging (fMRI) , 2004, IEEE Transactions on Biomedical Engineering.

[33]  John J. Foxe,et al.  Independent Brain Computer Interface Control using Visual Spatial Attention-Dependent Modulations of Parieto-occipital Alpha , 2005, Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering, 2005..

[34]  John J. Foxe,et al.  Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  W. Penfield,et al.  Electrocorticograms in man: Effect of voluntary movement upon the electrical activity of the precentral gyrus , 2005, Archiv für Psychiatrie und Nervenkrankheiten.

[36]  Thilo Hinterberger,et al.  An Auditory Brain-Computer Interface Based on the Self-Regulation of Slow Cortical Potentials , 2005, Neurorehabilitation and neural repair.

[37]  Hans Berger,et al.  Das Elektrenkephalogramm des Menschen , 1935, Naturwissenschaften.

[38]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[39]  G. Pfurtscheller,et al.  Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  Giuseppe Oriolo,et al.  The ASPICE project: inclusive design for the motor disabled , 2006, AVI '06.

[41]  N. Birbaumer Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[42]  G. Plourde Auditory evoked potentials. , 2006, Best practice & research. Clinical anaesthesiology.

[43]  F. Piccione,et al.  P300-based brain computer interface: Reliability and performance in healthy and paralysed participants , 2006, Clinical Neurophysiology.

[44]  Febo Cincotti,et al.  ResearchArticle Vibrotactile Feedback for Brain-Computer Interface Operation , 2007 .

[45]  N. Thakor,et al.  Journal of Neuroengineering and Rehabilitation Open Access a Brain-computer Interface with Vibrotactile Biofeedback for Haptic Information , 2007 .

[46]  José del R. Millán,et al.  Context-Based Filtering for Assisted Brain-Actuated Wheelchair Driving , 2007, Comput. Intell. Neurosci..

[47]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[48]  Yijun Wang,et al.  A Brain-Computer Interface Based on Multi-Modal Attention , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[49]  Benjamin Blankertz,et al.  A Note on Brain Actuated Spelling with the Berlin Brain-Computer Interface , 2007, HCI.

[50]  Febo Cincotti,et al.  Vibrotactile Feedback for Brain-Computer Interface Operation , 2007, Comput. Intell. Neurosci..

[51]  José del R. Millán,et al.  Evaluation Criteria for BCI Research , 2007 .

[52]  Constantine Stephanidis,et al.  Universal Access in Human-Computer Interaction. Ambient Interaction, 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007 Held as Part of HCI International 2007 Beijing, China, July 22-27, 2007 Proceedings, Part II , 2007, HCI.

[53]  J. Wolpaw,et al.  Towards an independent brain–computer interface using steady state visual evoked potentials , 2008, Clinical Neurophysiology.

[54]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[55]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[56]  Ko-ichiro Miyamoto,et al.  A brain-computer interface (BCI) system based on auditory stream segregation , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[57]  Cuntai Guan,et al.  Asynchronous P300-Based Brain--Computer Interfaces: A Computational Approach With Statistical Models , 2008, IEEE Transactions on Biomedical Engineering.

[58]  L. Cohen,et al.  Brain–computer interface in paralysis , 2008, Current opinion in neurology.

[59]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[60]  Febo Cincotti,et al.  Standard Model, File Formats and Methods in Brain-Computer Interface Research: Why? , 2009 .

[61]  C. Neuper,et al.  Toward a high-throughput auditory P300-based brain–computer interface , 2009, Clinical Neurophysiology.

[62]  A. Kübler,et al.  A Brain–Computer Interface Controlled Auditory Event‐Related Potential (P300) Spelling System for Locked‐In Patients , 2009, Annals of the New York Academy of Sciences.

[63]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[64]  Héctor Pomares,et al.  Evidences of cognitive effects over auditory steady-state responses by means of artificial neural networks and its use in brain-computer interfaces , 2009, Neurocomputing.

[65]  Benjamin Blankertz,et al.  Designing for uncertain, asymmetric control: Interaction design for brain-computer interfaces , 2009, Int. J. Hum. Comput. Stud..

[66]  Tom Heskes,et al.  Selecting features for BCI control based on a covert spatial attention paradigm , 2009, Neural Networks.

[67]  J. Wolpaw,et al.  Clinical Applications of Brain-Computer Interfaces: Current State and Future Prospects , 2009, IEEE Reviews in Biomedical Engineering.

[68]  Michael Bensch,et al.  Design and Implementation of a P300-Based Brain-Computer Interface for Controlling an Internet Browser , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[69]  S. Woolley,et al.  Insight in ALS: Awareness of behavioral change in patients with and without FTD , 2010, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[70]  Jan B. F. van Erp,et al.  A Tactile P300 Brain-Computer Interface , 2010, Front. Neurosci..

[71]  A. Cichocki,et al.  Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives , 2010, Progress in Neurobiology.

[72]  N. Birbaumer,et al.  An auditory oddball brain–computer interface for binary choices , 2010, Clinical Neurophysiology.

[73]  Jonathan R Wolpaw,et al.  A brain-computer interface for long-term independent home use , 2010, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[74]  G Pfurtscheller,et al.  Toward a hybrid brain–computer interface based on imagined movement and visual attention , 2010, Journal of neural engineering.

[75]  J. Wolpaw,et al.  Does the ‘P300’ speller depend on eye gaze? , 2010, Journal of neural engineering.

[76]  Benjamin Blankertz,et al.  A novel brain-computer interface based on the rapid serial visual presentation paradigm , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[77]  L. R. Quitadamo,et al.  Which Physiological Components are More Suitable for Visual ERP Based Brain–Computer Interface? A Preliminary MEG/EEG Study , 2010, Brain Topography.

[78]  B. Blankertz,et al.  (C)overt attention and visual speller design in an ERP-based brain-computer interface , 2010, Behavioral and Brain Functions.

[79]  A. Engel,et al.  An independent brain–computer interface using covert non-spatial visual selective attention , 2010, Journal of neural engineering.

[80]  Shangkai Gao,et al.  An online brain–computer interface using non-flashing visual evoked potentials , 2010, Journal of neural engineering.

[81]  A. Kübler,et al.  Motivation modulates the P300 amplitude during brain–computer interface use , 2010, Clinical Neurophysiology.

[82]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[83]  B. Blankertz,et al.  A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue , 2010, PloS one.

[84]  Shangkai Gao,et al.  An Auditory Brain–Computer Interface Using Active Mental Response , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[85]  M S Treder,et al.  Gaze-independent brain–computer interfaces based on covert attention and feature attention , 2011, Journal of neural engineering.

[86]  Benjamin Blankertz,et al.  A gaze independent spelling based on rapid serial visual presentation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[87]  Ali Bahramisharif,et al.  Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention , 2011, Journal of NeuroEngineering and Rehabilitation.

[88]  José del R. Millán,et al.  Brain-controlled telepresence robot by motor-disabled people , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[89]  F Babiloni,et al.  P300-based brain–computer interface for environmental control: an asynchronous approach , 2011, Journal of neural engineering.

[90]  A. Kübler,et al.  Flashing characters with famous faces improves ERP-based brain–computer interface performance , 2011, Journal of neural engineering.

[91]  M. Tangermann,et al.  Listen, You are Writing! Speeding up Online Spelling with a Dynamic Auditory BCI , 2011, Front. Neurosci..

[92]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[93]  Donatella Mattia,et al.  A Brain-Computer Interface as Input Channel for a Standard Assistive Technology Software , 2011, Clinical EEG and neuroscience.

[94]  Nico M Schmidt,et al.  Online detection of error-related potentials boosts the performance of mental typewriters , 2012, BMC Neuroscience.

[95]  F Cincotti,et al.  Workload measurement in a communication application operated through a P300-based brain–computer interface , 2011, Journal of neural engineering.

[96]  Benjamin Blankertz,et al.  A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System , 2011, Front. Neurosci..

[97]  Chang-Hwan Im,et al.  Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing , 2011, Journal of Neuroscience Methods.

[98]  A Belitski,et al.  P300 audio-visual speller , 2011, Journal of neural engineering.

[99]  Febo Cincotti,et al.  Out of the frying pan into the fire--the P300-based BCI faces real-world challenges. , 2011, Progress in brain research.

[100]  Moritz Grosse-Wentrup,et al.  Critical issues in state-of-the-art brain–computer interface signal processing , 2011, Journal of neural engineering.

[101]  D. Hu,et al.  Gaze independent brain–computer speller with covert visual search tasks , 2011, Clinical Neurophysiology.

[102]  B. Schölkopf,et al.  An online brain-computer interface based on shifting attention to concurrent streams of auditory stimuli. , 2012, Journal of neural engineering.

[103]  F. Babiloni,et al.  A covert attention P300-based brain–computer interface: Geospell , 2012, Ergonomics.

[104]  Jonathan R. Wolpaw,et al.  Brain–Computer InterfacesPrinciples and Practice , 2012 .

[105]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[106]  Peter Fischer,et al.  The self-assembly, aggregation and phase transitions of food protein systems in one, two and three dimensions , 2013, Reports on progress in physics. Physical Society.