Affective brain-computer interfaces: Psychophysiological markers of emotion in healthy persons and in persons with amyotrophic lateral sclerosis

Affective Brain-Computer Interfaces (BCI) are systems that measure signals from the peripheral and central nervous system, extract features related to affective states of the user, and use these features to adapt human-computer interaction (HCI). Affective BCIs provide new perspectives on the applicability of BCIs. Affective BCIs may serve as assessment tools and adaptive systems for HCI for the general population and may prove to be especially interesting for people with severe motor impairment. In this context, affective BCIs will enable simultaneous expression of affect and content, thus providing more quality of life for the patient and the caregiver. In the present paper, we will present psychophysiological markers for affective BCIs, and discuss their usability in the day to day life of patients with amyotrophic lateral sclerosis (ALS).

[1]  Rosalind W. Picard,et al.  Signal processing for recognition of human frustration , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[2]  Giuseppe Oriolo,et al.  The ASPICE project: inclusive design for the motor disabled , 2006, AVI '06.

[3]  N. Birbaumer.  Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[4]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[5]  A. Damasio,et al.  Basic emotions are associated with distinct patterns of cardiorespiratory activity. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  M. Arns,et al.  Efficacy of Neurofeedback Treatment in ADHD: The Effects on Inattention, Impulsivity and Hyperactivity: A Meta-Analysis , 2009, Clinical EEG and neuroscience.

[7]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[8]  P. Kleinginna,et al.  A categorized list of emotion definitions, with suggestions for a consensual definition , 1981 .

[9]  Agneta H. Fischer,et al.  Feelings and Emotions: The Amsterdam Symposium , 2004 .

[10]  A. Jersild,et al.  Feelings and Emotions. , 1929 .

[11]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Thilo Hinterberger,et al.  Neurofeedback for Children with ADHD: A Comparison of SCP and Theta/Beta Protocols , 2007, Applied psychophysiology and biofeedback.

[13]  Graham Clarke,et al.  A user-independent real-time emotion recognition system for software agents in domestic environments , 2007, Eng. Appl. Artif. Intell..

[14]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[15]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[16]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[17]  B. Brooks,et al.  Clinical epidemiology of amyotrophic lateral sclerosis. , 1996, Neurologic clinics.

[18]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[19]  N. Birbaumer,et al.  Self-regulation of Slow Cortical Potentials: A New Treatment for Children With Attention-Deficit/Hyperactivity Disorder , 2006, Pediatrics.

[20]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[21]  ALBERTO C. SEGUIN.  Erasistratus, Antiochus, and Psychosomatic Medicine , 1948, Psychosomatic medicine.

[22]  Henrike Gappa,et al.  IPCA: Adaptive Interfaces Based upon Biofeedback Sensors , 2004, ICCHP.

[23]  Michael Siniatchkin,et al.  On the Pathophysiology of Migraine—Links for “Empirically Based Treatment” with Neurofeedback , 2002, Applied psychophysiology and biofeedback.

[24]  Scotty D. Craig,et al.  Integrating Affect Sensors in an Intelligent Tutoring System , 2004 .

[25]  G. R. Muller,et al.  Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment , 2003, Clinical Neurophysiology.

[26]  P. Ekman,et al.  Autonomic nervous system activity distinguishes among emotions. , 1983, Science.

[27]  J. Kassubek,et al.  Emotional responding in amyotrophic lateral sclerosis , 2005, Journal of Neurology.

[28]  Aribert Rothenberger,et al.  Training of slow cortical potentials in attention-deficit/hyperactivity disorder: evidence for positive behavioral and neurophysiological effects , 2004, Biological Psychiatry.

[29]  J. Cassisi,et al.  Neurofeedback Training for a Patient with Thalamic and Cortical Infarctions , 2003, Applied psychophysiology and biofeedback.

[30]  E. Vyzas,et al.  Affective Pattern Classification , 2002 .

[31]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[33]  J. Rabkin,et al.  Prevalence of depressive disorders and change over time in late-stage ALS , 2005, Neurology.

[34]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[35]  J. Singer,et al.  Cognitive, social, and physiological determinants of emotional state. , 1962 .

[36]  N Birbaumer,et al.  Negative potential shifts and the prediction of the outcome of neurofeedback therapy in epilepsy , 1999, Clinical Neurophysiology.

[37]  Tom Cochrane,et al.  Eight dimensions for the emotions , 2009 .

[38]  Desney S. Tan,et al.  Playing with your brain: brain-computer interfaces and games , 2007, ACE '07.

[39]  Seguin Ca.  Erasistratus, Antiochus, and psychosomatic medicine. , 1948 .

[40]  Tom E. Bishop,et al.  Blind Image Restoration Using a Block-Stationary Signal Model , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[41]  Edward Cutrell,et al.  BCI for passive input in HCI , 2007 .

[42]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999 .

[43]  P. Tonin,et al.  P300-Based Brain–Computer Interface Communication: Evaluation and Follow-up in Amyotrophic Lateral Sclerosis , 2009, Front. Neuropro..

[44]  E. Granholm,et al.  Pupillometric measures of cognitive and emotional processes. , 2004, International Journal of Psychophysiology.

[45]  L. F. Barrett,et al.  Handbook of Emotions , 1993 .

[46]  Hiroshi Mitsumoto,et al.  Predictors and course of elective long‐term mechanical ventilation: A prospective study of ALS patients , 2006, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[47]  M. Powell Lawton,et al.  Annual Review of Gerontology and Geriatrics , 1988, Springer Berlin Heidelberg.

[48]  K Wing,et al.  Effect of Neurofeedback on Motor Recovery of a Patient with Brain Injury: A Case Study and Its Implications for Stroke Rehabilitation , 2001, Topics in stroke rehabilitation.

[49]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[50]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[51]  Desney S. Tan,et al.  Brain-computer interfaces for hci and games , 2008, CHI Extended Abstracts.

[52]  N. Birbaumer,et al.  Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients? , 2008, Clinical Neurophysiology.

[53]  P. Ekman,et al.  Emotion in the Human Face: Guidelines for Research and an Integration of Findings , 1972 .

[54]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[55]  G. Kozlowski,et al.  Neurofeedback treatment of epilepsy. , 2005, Child and adolescent psychiatric clinics of North America.

[56]  R. Davidson.  Affective neuroscience and psychophysiology: toward a synthesis. , 2003, Psychophysiology.

[57]  Enrique Leon,et al.  Emotion-aware intelligent environments: A user perspective , 2009, Intelligent Environments.

[58]  J. Martínez-Selva,et al.  Psychophysiological, Behavioral, and Cognitive Indices of the Emotional Response: A Factor-Analytic Study , 2008, The Spanish Journal of Psychology.

[59]  P. Griffiths.  What emotions really are : the problem of psychological categories , 1997 .

[60]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[61]  M. Powell Lawton,et al.  Annual Review of Gerontology and Geriatrics , 1990, Springer Berlin Heidelberg.

[62]  José del R. Millán,et al.  Brain-Computer Interfaces for Communication and Motor Control—Perspectives on Clinical Applications , 2007 .

[63]  F. Piccione,et al.  P300-based brain computer interface: Reliability and performance in healthy and paralysed participants , 2006, Clinical Neurophysiology.

[64]  G. Rozelle,et al.  Neurotherapy for stroke rehabilitation: A single case study , 1995, Biofeedback and self-regulation.

[65]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[66]  E. Fetz,et al.  Operant Conditioning of Specific Patterns of Neural and Muscular Activity , 1971, Science.

[67]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[68]  E. Donchin,et al.  Brain-computer interface research at the university of south Florida cognitive psychophysiology laboratory: the P300 speller , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[69]  W. James,et al.  What Is an Emotion , 1977 .

[70]  Jonathan E Walker,et al.  Power Spectral Frequency and Coherence Abnormalities in Patients with Intractable Epilepsy and Their Usefulness in Long-Term Remediation of Seizures Using Neurofeedback , 2008, Clinical EEG and neuroscience.

[71]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[72]  W. Johnston,et al.  Correlates of suffering in amyotrophic lateral sclerosis , 1999, Neurology.

[73]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[74]  J. F. Dobson,et al.  Erasistratus , 1927, Proceedings of the Royal Society of Medicine.

[75]  E. Diener,et al.  Autonomic arousal feedback and emotional experience: evidence from the spinal cord injured. , 1988, Journal of personality and social psychology.

[76]  Hideaki Hayashi,et al.  ALS patients on TPPV , 2003, Neurology.

[77]  Carla H. Lagorio,et al.  Psychology , 1929, Nature.

[78]  Daniel J. Strauss,et al.  Neurofeedback by neural correlates of auditory selective attention as possible application for tinnitus therapies , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[79]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[80]  Melody Moore Jackson,et al.  A galvanic skin response interface for people with severe motor disabilities , 2004, Assets '04.

[81]  T. Surmeli,et al.  QEEG Guided Neurofeedback Therapy in Personality Disorders: 13 Case Studies , 2009, Clinical EEG and neuroscience.

[82]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999, Journal of personality and social psychology.

[83]  Ethan R. Buch,et al.  Think to Move: a Neuromagnetic Brain-Computer Interface (BCI) System for Chronic Stroke , 2008, Stroke.

[84]  Merit Cudkowicz,et al.  Measures and markers in Amyotrophic Lateral Sclerosis , 2004, NeuroRX.

[85]  Elias Vyzas.  Recognition of emotional and cognitive states using physiological data , 1999 .