Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond

In their early days, brain-computer interfaces (BCIs) were only considered as control channel for end users with severe motor impairments such as people in the locked-in state. But, thanks to the multidisciplinary progress achieved over the last decade, the range of BCI applications has been substantially enlarged. Indeed, today BCI technology cannot only translate brain signals directly into control signals, but also can combine such kind of artificial output with a natural muscle-based output. Thus, the integration of multiple biological signals for real-time interaction holds the promise to enhance a much larger population than originally thought end users with preserved residual functions who could benefit from new generations of assistive technologies. A BCI system that combines a BCI with other physiological or technical signals is known as hybrid BCI (hBCI). In this work, we review the work of a large scale integrated project funded by the European commission which was dedicated to develop practical hybrid BCIs and introduce them in various fields of applications. This article presents an hBCI framework, which was used in studies with nonimpaired as well as end users with motor impairments.

[1]  Mahoney Fi,et al.  FUNCTIONAL EVALUATION: THE BARTHEL INDEX. , 1965 .

[2]  F. I. Mahonery Functional evaluation : Barthel index , 1965 .

[3]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[4]  F. Babiloni,et al.  Adaptive brain interfaces for physically-disabled people , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[5]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[6]  M. Popovic,et al.  Clinical evaluation of the bionic glove. , 1999, Archives of physical medicine and rehabilitation.

[7]  G. Pfurtscheller,et al.  EEG-based communication: presence of an error potential , 2000, Clinical Neurophysiology.

[8]  Jukka Heikkonen,et al.  Local Neural Classifier for EEG-Based Recognition of Mental Tasks , 2000, IJCNN.

[9]  J. Wolpaw,et al.  Brain-computer communication: unlocking the locked in. , 2001, Psychological bulletin.

[10]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[11]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[12]  Jukka Heikkonen,et al.  A local neural classifier for the recognition of EEG patterns associated to mental tasks , 2002, IEEE Trans. Neural Networks.

[13]  K.-R. Muller,et al.  Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  J. Mourino,et al.  Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Hendrik Van Brussel,et al.  Shared control for intelligent wheelchairs: an implicit estimation of the user intention , 2003 .

[16]  G. Alon,et al.  Persons with C5 or C6 tetraplegia achieve selected functional gains using a neuroprosthesis. , 2003, Archives of physical medicine and rehabilitation.

[17]  G.E. Birch,et al.  A general framework for brain-computer interface design , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  G. Pfurtscheller,et al.  ‘Thought’ – control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia , 2003, Neuroscience Letters.

[19]  T. Sinkjaer,et al.  Clinical evaluation of Functional Electrical Therapy in acute hemiplegic subjects. , 2003, Journal of rehabilitation research and development.

[20]  N. Dimitrova,et al.  Interpretation of EMG changes with fatigue: facts, pitfalls, and fallacies. , 2003, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[21]  José del R. Millán,et al.  Adaptive brain interfaces , 2003, Commun. ACM.

[22]  Klaus-Robert Müller,et al.  Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms , 2004, IEEE Transactions on Biomedical Engineering.

[23]  José del R. Millán,et al.  Brain-actuated interaction , 2004, Artif. Intell..

[24]  Maarten J. IJzerman,et al.  Survey of the needs of patients with spinal cord injury: impact and priority for improvement in hand function in tetraplegics , 2004, Spinal Cord.

[25]  Galina L. Rogova,et al.  Reliability In Information Fusion : Literature Survey , 2004 .

[26]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[27]  N. Birbaumer,et al.  Predictability of Brain-Computer Communication , 2004 .

[28]  G. Pfurtscheller,et al.  EEG-based neuroprosthesis control: A step towards clinical practice , 2005, Neuroscience Letters.

[29]  Paul C. Schutte,et al.  The H-Metaphor as a Guideline for Vehicle Automation and Interaction , 2005 .

[30]  A. Buttfield,et al.  Towards a robust BCI: error potentials and online learning , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  G. Pfurtscheller,et al.  Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. , 2006, Biomedizinische Technik. Biomedical engineering.

[32]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[33]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  José del R. Millán,et al.  Context-Based Filtering for Assisted Brain-Actuated Wheelchair Driving , 2007, Comput. Intell. Neurosci..

[35]  K. Müller,et al.  Single Trial Classification of Motor Imagination Using 6 Dry EEG Electrodes , 2007, PloS one.

[36]  Jonathan R Wolpaw,et al.  Brain–computer interfaces as new brain output pathways , 2007, The Journal of physiology.

[37]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[38]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[39]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[40]  José del R. Millán,et al.  Simultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy , 2008 .

[41]  M. Nuttin,et al.  A brain-actuated wheelchair: Asynchronous and non-invasive Brain–computer interfaces for continuous control of robots , 2008, Clinical Neurophysiology.

[42]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[43]  Ricardo Chavarriaga,et al.  Non-Invasive Brain-Machine Interaction , 2008, Int. J. Pattern Recognit. Artif. Intell..

[44]  K. Müller,et al.  Finding stationary subspaces in multivariate time series. , 2009, Physical review letters.

[45]  M. Nuttin,et al.  Asynchronous non-invasive brain-actuated control of an intelligent wheelchair , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[46]  G. Pfurtscheller,et al.  Could the beta rebound in the EEG be suitable to realize a “brain switch”? , 2009, Clinical Neurophysiology.

[47]  P. Langhorne,et al.  Motor recovery after stroke: a systematic review , 2009, The Lancet Neurology.

[48]  Tobias Seidl,et al.  Validation of brain-machine interfaces during parabolic flight. , 2009, International review of neurobiology.

[49]  Brice Rebsamen,et al.  A brain controlled wheelchair to navigate in familiar environments. , 2010, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[50]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[51]  Jonathan R Wolpaw,et al.  A brain-computer interface for long-term independent home use , 2010, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[52]  Anatole Lécuyer,et al.  Exploring Large Virtual Environments by Thoughts Using a BrainComputer Interface Based on Motor Imagery and High-Level Commands , 2010, PRESENCE: Teleoperators and Virtual Environments.

[53]  José del R. Millán,et al.  Brain–computer interfaces for space applications , 2011, Personal and Ubiquitous Computing.

[54]  Klaus-Robert Müller,et al.  A regularized discriminative framework for EEG analysis with application to brain–computer interface , 2010, NeuroImage.

[55]  Stefan Haufe,et al.  The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology , 2010, Front. Neurosci..

[56]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[57]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[58]  Febo Cincotti,et al.  Asynchronous P300-Based Brain-Computer Interface to Control a Virtual Environment: Initial Tests on End Users , 2011, Clinical EEG and neuroscience.

[59]  Ricardo Chavarriaga,et al.  A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities , 2011, Journal of neural engineering.

[60]  José del R. Millán,et al.  Are we ready? Issues in transferring BCI technology from experts to users , 2011 .

[61]  A. Kübler,et al.  Flashing characters with famous faces improves ERP-based brain–computer interface performance , 2011, Journal of neural engineering.

[62]  L. Cohen,et al.  Neuroplasticity in the context of motor rehabilitation after stroke , 2011, Nature Reviews Neurology.

[63]  Donatella Mattia,et al.  A Brain-Computer Interface as Input Channel for a Standard Assistive Technology Software , 2011, Clinical EEG and neuroscience.

[64]  Febo Cincotti,et al.  Tools for Brain-Computer Interaction: A General Concept for a Hybrid BCI , 2011, Front. Neuroinform..

[65]  F Cincotti,et al.  Workload measurement in a communication application operated through a P300-based brain–computer interface , 2011, Journal of neural engineering.

[66]  F. Cincotti,et al.  Can severe Acquired Brain Injury users control a communication application operated through a P300-based brain computer interface? , 2011 .

[67]  C. Grozea,et al.  Bristle-sensors—low-cost flexible passive dry EEG electrodes for neurofeedback and BCI applications , 2011, Journal of neural engineering.

[68]  Klaus-Robert Müller,et al.  Introduction to machine learning for brain imaging , 2011, NeuroImage.

[69]  Sebastian Bosse,et al.  Toward a Direct Measure of Video Quality Perception Using EEG , 2012, IEEE Transactions on Image Processing.

[70]  S. Debener,et al.  How about taking a low-cost, small, and wireless EEG for a walk? , 2012, Psychophysiology.

[71]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[72]  Dean J Krusienski,et al.  Brain-computer interfaces in medicine. , 2012, Mayo Clinic proceedings.

[73]  Brendan Z. Allison,et al.  Comparison of Dry and Gel Based Electrodes for P300 Brain–Computer Interfaces , 2012, Front. Neurosci..

[74]  M. Molinari,et al.  Brain Computer Interface for Hand Motor Function Restoration and Rehabilitation , 2012 .

[75]  Ricardo Chavarriaga,et al.  Detection of anticipatory brain potentials during car driving , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[76]  Christa Neuper,et al.  Proposing a Standardized Protocol for Raw Biosignal Transmission , 2012, IEEE Transactions on Biomedical Engineering.

[77]  Anton Nijholt,et al.  Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications , 2012 .

[78]  K. Müller,et al.  Single-trial analysis of the neural correlates of speech quality perception , 2013, Journal of neural engineering.

[79]  Vera Kaiser,et al.  Thinking Penguin: Multimodal Brain–Computer Interface Control of a VR Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[80]  Zhaohui Wu,et al.  The Convergence of Machine and Biological Intelligence , 2013, IEEE Intelligent Systems.

[81]  José del R. Millán,et al.  Brain-Controlled Wheelchairs: A Robotic Architecture , 2013, IEEE Robotics & Automation Magazine.

[82]  José del R. Millán,et al.  Transferring brain-computer interfaces beyond the laboratory: Successful application control for motor-disabled users , 2013, Artif. Intell. Medicine.

[83]  Reza Fazel-Rezai,et al.  A Review of Hybrid Brain-Computer Interface Systems , 2013, Adv. Hum. Comput. Interact..

[84]  Vera Kaiser,et al.  Hybrid brain-computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with high-level spinal cord injury , 2013, Artif. Intell. Medicine.

[85]  Lucian Gheorghe,et al.  Steering timing prediction in a driving simulator task , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[86]  J. Krakauer,et al.  The interaction between training and plasticity in the poststroke brain. , 2013, Current opinion in neurology.

[87]  Klaus-Robert Müller,et al.  Motor Imagery for Severely Motor-Impaired Patients: Evidence for Brain-Computer Interfacing as Superior Control Solution , 2014, PloS one.

[88]  E-J Hoogerwerf,et al.  Clinical evaluation of BrainTree, a motor imagery hybrid BCI speller , 2014, Journal of neural engineering.

[89]  Tom Chau,et al.  A case study of linear classifiers adapted using imperfect labels derived from human event-related potentials , 2014, Pattern Recognit. Lett..