Multi-modal Affect Induction for Affective Brain-Computer Interfaces

Reliable applications of affective brain-computer interfaces (aBCI) in realistic, multi-modal environments require a detailed understanding of the processes involved in emotions. To explore the modalityspecific nature of affective responses, we studied neurophysiological responses (i.e., EEG) of 24 participants during visual, auditory, and audiovisual affect stimulation. The affect induction protocols were validated by participants' subjective ratings and physiological responses (i.e., ECG). Coherent with literature, we found modality-specific responses in the EEG: posterior alpha power decreases during visual stimulation and increases during auditory stimulation, anterior alpha power tends to decrease during auditory stimulation and to increase during visual stimulation. We discuss the implications of these results for multimodal aBCI.

[1]  L. Aftanas,et al.  Analysis of Evoked EEG Synchronization and Desynchronization in Conditions of Emotional Activation in Humans: Temporal and Topographic Characteristics , 2004, Neuroscience and Behavioral Physiology.

[2]  Gilles Pourtois,et al.  Dynamics of emotional effects on spatial attention in the human visual cortex. , 2006, Progress in brain research.

[3]  T. Ethofer,et al.  Decoding of emotional information in voice-sensitive cortices , 2009, NeuroImage.

[4]  J. Panksepp,et al.  Emotional sounds and the brain: the neuro-affective foundations of musical appreciation , 2002, Behavioural Processes.

[5]  P. Vuilleumier,et al.  How brains beware: neural mechanisms of emotional attention , 2005, Trends in Cognitive Sciences.

[6]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  N. Hadjikhani,et al.  Seeing Fearful Body Expressions Activates the Fusiform Cortex and Amygdala , 2003, Current Biology.

[8]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  K. Scherer,et al.  The voices of wrath: brain responses to angry prosody in meaningless speech , 2005, Nature Neuroscience.

[10]  David Sander,et al.  A systems approach to appraisal mechanisms in emotion , 2005, Neural Networks.

[11]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[12]  G. Hajcak,et al.  Event-Related Potentials, Emotion, and Emotion Regulation: An Integrative Review , 2010, Developmental neuropsychology.

[13]  M. Bradley,et al.  Affective reactions to acoustic stimuli. , 2000, Psychophysiology.

[14]  Lisa Feldman Barrett,et al.  Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies , 2008, NeuroImage.

[15]  Mikko Sams,et al.  Emotion Modulates Early Auditory Response to Speech , 2009, Journal of Cognitive Neuroscience.

[16]  J. Mäkelä,et al.  Magnetoencephalographic cortical rhythms. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[17]  Aina Puce,et al.  Audiovisual Non-Verbal Dynamic Faces Elicit Converging fMRI and ERP Responses , 2009, Brain Topography.

[18]  P. Lang.  International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[19]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[21]  Erol Basar,et al.  Emotional face expressions are differentiated with brain oscillations. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[23]  Vera Ferrari,et al.  Repetitive picture processing: Autonomic and cortical correlates , 2006, Brain Research.

[24]  R. Näätänen,et al.  The mismatch negativity (MMN) in basic research of central auditory processing: A review , 2007, Clinical Neurophysiology.

[25]  O. Jensen,et al.  Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition , 2010, Front. Hum. Neurosci..

[26]  Christina M Krause,et al.  Cognition- and memory-related ERD/ERS responses in the auditory stimulus modality. , 2006, Progress in brain research.

[27]  M. Bradley,et al.  Emotional perception: correlation of functional MRI and event-related potentials. , 2006, Cerebral cortex.