Brain–computer interfaces for communication with nonresponsive patients

A substantial number of patients who survive severe brain injury progress to a nonresponsive state of wakeful unawareness, referred to as a vegetative state (VS). They appear to be awake, but show no signs of awareness of themselves, or of their environment in repeated clinical examinations. However, recent neuroimaging research demonstrates that some VS patients can respond to commands by willfully modulating their brain activity according to instruction. Brain–computer interfaces (BCIs) may allow such patients to circumvent the barriers imposed by their behavioral limitations and communicate with the outside world. However, although such devices would undoubtedly improve the quality of life for some patients and their families, developing BCI systems for behaviorally nonresponsive patients presents substantial technical and clinical challenges. Here we review the state of the art of BCI research across noninvasive neuroimaging technologies, and propose how such systems should be developed further to provide fully fledged communication systems for behaviorally nonresponsive populations. Ann Neurol 2012;72:312–323

[1]  M. McCabe Presidential address. , 1970, Irish nurses' journal.

[2]  E. Donchin,et al.  On quantifying surprise: the variation of event-related potentials with subjective probability. , 1977, Psychophysiology.

[3]  William L. Wilkie,et al.  Presidential Address: 1980 , 1981 .

[4]  E. Donchin Presidential address, 1980. Surprise!...Surprise? , 1981, Psychophysiology.

[5]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[6]  Donald G. Childers,et al.  Brain potentials related to seeing one's own name , 1987, Brain and Language.

[7]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[8]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[9]  H. Herzog,et al.  Frontal lobe function in amyotrophic lateral sclerosis: a neuropsychologic and positron emission tomography study , 1992, Acta neurologica Scandinavica.

[10]  R. Turner,et al.  Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Ravi S. Menon,et al.  Functional brain mapping by blood oxygenation level-dependent contrast magnetic resonance imaging. A comparison of signal characteristics with a biophysical model. , 1993, Biophysical journal.

[12]  N. Childs,et al.  Accuracy of diagnosis of persistent vegetative state , 1993, Neurology.

[13]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited , 1995, NeuroImage.

[14]  S. Ashwal,et al.  Medical aspects of the persistent vegetative state--a correction. The Multi-Society Task Force on PVS. , 1995, The New England journal of medicine.

[15]  L. Goldstein,et al.  Cognitive deficits in non-demented amyotrophic lateral sclerosis patients: a neuropsychological investigation , 1995, Journal of the Neurological Sciences.

[16]  K. Andrews,et al.  Misdiagnosis of the vegetative state: retrospective study in a rehabilitation unit , 1996, BMJ.

[17]  K. Abe,et al.  Cognitive function in amyotrophic lateral sclerosis , 1997, Journal of the Neurological Sciences.

[18]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[19]  N. Birbaumer,et al.  [What are you doing when you are doing nothing? ERP components without a cognitive task]. , 1997, Zeitschrift fur experimentelle Psychologie : Organ der Deutschen Gesellschaft fur Psychologie.

[20]  [Locked-in: imprisoned in your own body]. , 1998, Pflege aktuell.

[21]  [Locked-in: imprisoned in your own body]. , 1998 .

[22]  J D Pickard,et al.  Cortical processing in persistent vegetative state. Wolfson Brain Imaging Centre Team. , 1998, Lancet.

[23]  J. Meilof Cortical processing in persistent vegetative state , 1998, The Lancet.

[24]  M. Erb,et al.  Activation of Cortical and Cerebellar Motor Areas during Executed and Imagined Hand Movements: An fMRI Study , 1999, Journal of Cognitive Neuroscience.

[25]  S. Butler,et al.  Coma Outcome Prediction Using Event-Related Potentials: P3 and Mismatch Negativity , 2000, Audiology and Neurotology.

[26]  Willy Gsell,et al.  The use of cerebral blood flow as an index of neuronal activity in functional neuroimaging: experimental and pathophysiological considerations , 2000, Journal of Chemical Neuroanatomy.

[27]  J. Adams,et al.  The neuropathology of the vegetative state after an acute brain insult. , 2000, Brain : a journal of neurology.

[28]  H. Shevrin,et al.  Event-related markers of unconscious processes. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[29]  Fabien Perrin,et al.  Semantic analysis of auditory input during sleep: studies with event related potentials. , 2002, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  S. Scott,et al.  Detecting Residual Cognitive Function in Persistent Vegetative State , 2002, Neurocase.

[31]  J. Giacino,et al.  The minimally conscious state: Definition and diagnostic criteria , 2002, Neurology.

[32]  D. Shewmon,et al.  The minimally conscious state: definition and diagnostic criteria. , 2002, Neurology.

[33]  N. Birbaumer,et al.  Stimulus complexity enhances auditory discrimination in patients with extremely severe brain injuries , 2003, Neuroscience Letters.

[34]  F. Cincotti,et al.  The use of EEG modifications due to motor imagery for brain-computer interfaces , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Steven Laureys,et al.  Brain function in coma, vegetative state, and related disorders , 2004, The Lancet Neurology.

[36]  P M Rossini,et al.  Does cerebrovascular disease affect the coupling between neuronal activity and local haemodynamics? , 2004, Brain : a journal of neurology.

[37]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[38]  R. Munday,et al.  The sensory modality assessment and rehabilitation technique (SMART): a valid and reliable assessment for vegetative state and minimally conscious state patients , 2004, Brain injury.

[39]  Soo-Young Lee,et al.  Brain–computer interface using fMRI: spatial navigation by thoughts , 2004, Neuroreport.

[40]  F. Gerstenbrand,et al.  Varieties of the locked-in syndrome , 1979, Journal of Neurology.

[41]  Matthew H. Davis,et al.  Using a hierarchical approach to investigate residual auditory cognition in persistent vegetative state. , 2005, Progress in brain research.

[42]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Caroline Schnakers,et al.  Bispectral analysis of electroencephalogram signals during recovery from coma: Preliminary findings , 2005, Neuropsychological rehabilitation.

[44]  J. Wolpaw,et al.  Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface , 2005, Neurology.

[45]  N. Birbaumer,et al.  Information processing in severe disorders of consciousness: Vegetative state and minimally conscious state , 2005, Clinical Neurophysiology.

[46]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[47]  Manuel Schabus,et al.  Brain response to one's own name in vegetative state, minimally conscious state, and locked-in syndrome. , 2006, Archives of neurology.

[48]  N. Ramsey,et al.  Towards human BCI applications based on cognitive brain systems: an investigation of neural signals recorded from the dorsolateral prefrontal cortex , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[50]  D. Boas,et al.  Diffuse optical tomography system to image brain activation with improved spatial resolution and validation with functional magnetic resonance imaging. , 2006, Applied optics.

[51]  Niels Birbaumer,et al.  fMRI Brain-Computer Interface: A Tool for Neuroscientific Research and Treatment , 2007, Comput. Intell. Neurosci..

[52]  Steven Laureys,et al.  When thoughts become action: An fMRI paradigm to study volitional brain activity in non-communicative brain injured patients , 2007, NeuroImage.

[53]  F. Irani,et al.  Functional Near Infrared Spectroscopy (fNIRS): An Emerging Neuroimaging Technology with Important Applications for the Study of Brain Disorders , 2007, The Clinical neuropsychologist.

[54]  A. Kübler,et al.  Brain–computer interfaces in the continuum of consciousness , 2007, Current opinion in neurology.

[55]  Steven Laureys,et al.  Locked-in: don’t judge a book by its cover , 2007, Journal of Neurology, Neurosurgery, and Psychiatry.

[56]  Rajesh P. N. Rao,et al.  Real-time functional brain mapping using electrocorticography , 2007, NeuroImage.

[57]  Matthew H. Davis,et al.  Do vegetative patients retain aspects of language comprehension? Evidence from fMRI. , 2007, Brain : a journal of neurology.

[58]  Masashi Kiguchi,et al.  A Communication Means for Totally Locked-in ALS Patients Based on Changes in Cerebral Blood Volume Measured with Near-Infrared Light , 2007, IEICE Trans. Inf. Syst..

[59]  R. DeCharms,et al.  Reading and controlling human brain activation using real-time functional magnetic resonance imaging , 2007, Trends in Cognitive Sciences.

[60]  Xiaoping P. Hu,et al.  Real‐time fMRI using brain‐state classification , 2007, Human brain mapping.

[61]  N. Birbaumer,et al.  Behavioral and Brain Functions , 2008 .

[62]  Dominique Morlet,et al.  Novelty P3 elicited by the subject’s own name in comatose patients , 2008, Clinical Neurophysiology.

[63]  Adrian M Owen,et al.  Using neuroimaging to detect awareness in disorders of consciousness. , 2008, Functional neurology.

[64]  N. Birbaumer,et al.  A brain–computer interface tool to assess cognitive functions in completely paralyzed patients with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[65]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[66]  Matthew H. Davis,et al.  Detecting Awareness in the Vegetative State , 2006, Science.

[67]  R. DeCharms Applications of real-time fMRI , 2008, Nature Reviews Neuroscience.

[68]  N. Birbaumer,et al.  Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients? , 2008, Clinical Neurophysiology.

[69]  Steve Majerus,et al.  Cognitive function in the locked-in syndrome , 2008, Journal of Neurology.

[70]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[71]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[72]  S Laureys,et al.  Voluntary brain processing in disorders of consciousness , 2008, Neurology.

[73]  M. Boly,et al.  Diagnostic accuracy of the vegetative and minimally conscious state: Clinical consensus versus standardized neurobehavioral assessment , 2009, BMC neurology.

[74]  A. Owen,et al.  Neuroimaging and the Vegetative State , 2009, Annals of the New York Academy of Sciences.

[75]  F. Guenther,et al.  A Wireless Brain-Machine Interface for Real-Time Speech Synthesis , 2009, PloS one.

[76]  C. Neuper,et al.  Toward a high-throughput auditory P300-based brain–computer interface , 2009, Clinical Neurophysiology.

[77]  Matthew H. Davis,et al.  Towards the routine use of brain imaging to aid the clinical diagnosis of disorders of consciousness. , 2009, Brain : a journal of neurology.

[78]  A Kübler,et al.  Life can be worth living in locked-in syndrome. , 2009, Progress in brain research.

[79]  Jong-Hwan Lee,et al.  Automated classification of fMRI data employing trial-based imagery tasks , 2009, Medical Image Anal..

[80]  A. Kübler,et al.  A Brain–Computer Interface Controlled Auditory Event‐Related Potential (P300) Spelling System for Locked‐In Patients , 2009, Annals of the New York Academy of Sciences.

[81]  Bettina Sorger,et al.  Another kind of 'BOLD Response': answering multiple-choice questions via online decoded single-trial brain signals. , 2009, Progress in brain research.

[82]  Caroline Schnakers,et al.  Behavioral assessment in patients with disorders of consciousness: gold standard or fool's gold? , 2009, Progress in brain research.

[83]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[84]  F. Jolesz,et al.  Brain–machine interface via real-time fMRI: Preliminary study on thought-controlled robotic arm , 2009, Neuroscience Letters.

[85]  Werner Lutzenberger,et al.  Recognition of affective prosody in brain-damaged patients and healthy controls: A neurophysiological study using EEG and whole-head MEG , 2009, Cognitive, affective & behavioral neuroscience.

[86]  A. Owen,et al.  A multimodal approach to the assessment of patients with disorders of consciousness. , 2009, Progress in brain research.

[87]  Steve Majerus,et al.  Detecting Consciousness in a Total Locked-in Syndrome: an Active Event-related Paradigm Detecting Consciousness in a Total Lis , 2022 .

[88]  S. Gielen,et al.  The brain–computer interface cycle , 2009, Journal of neural engineering.

[89]  Fang Sun,et al.  Willful modulation of brain activity in disorders of consciousness. , 2010, The New England journal of medicine.

[90]  D. Hassabis,et al.  Decoding Individual Episodic Memory Traces in the Human Hippocampus , 2010, Current Biology.

[91]  Walter G Sannita,et al.  Unresponsive wakefulness syndrome: a new name for the vegetative state or apallic syndrome , 2010, BMC medicine.

[92]  Nicholas D. Schiff,et al.  In the Blink of the Mind’s Eye , 2010, The Hastings Center report.

[93]  Jonathan D. Victor,et al.  Determination of awareness in patients with severe brain injury using EEG power spectral analysis , 2011, Clinical Neurophysiology.

[94]  Niels Birbaumer,et al.  Real-time support vector classification and feedback of multiple emotional brain states , 2011, NeuroImage.

[95]  Mélanie Boly,et al.  Measuring the fading consciousness in the human brain. , 2011, Current opinion in neurology.

[96]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[97]  Febo Cincotti,et al.  Out of the frying pan into the fire--the P300-based BCI faces real-world challenges. , 2011, Progress in brain research.

[98]  N. Schiff,et al.  Dissociations between behavioural and functional magnetic resonance imaging-based evaluations of cognitive function after brain injury. , 2011, Brain : a journal of neurology.

[99]  Srivas Chennu,et al.  Bedside detection of awareness in the vegetative state: a cohort study , 2011, The Lancet.

[100]  Steven Laureys,et al.  Coma and consciousness: Paradigms (re)framed by neuroimaging , 2012, NeuroImage.

[101]  Nicholas D Schiff,et al.  Pattern classification of volitional functional magnetic resonance imaging responses in patients with severe brain injury. , 2012, Archives of neurology.

[102]  John D Pickard,et al.  Visual cognition in disorders of consciousness: From V1 to top‐down attention , 2013, Human brain mapping.