Interactive component extraction from fEEG, fNIRS and peripheral biosignals for affective brain-machine interfacing paradigms

This paper investigates whether some well understood principles of human behavioral analysis can be used to design novel paradigms for affective brain-computer/machine interfaces. This is achieved by using the visual, audio, and audiovisual stimuli representing human emotions. The analysis of brain responses to such stimuli involves several challenges related to the conditioning of brain electrical responses, extraction of the responses to stimuli and mutual information between the several physiological recording modalities used. This is achieved in the time-frequency domain, using multichannel empirical mode decomposition (EMD), which proves very accurate in the joint analysis of neurophysiological and peripheral body signals. Our results indicate the usefulness of such an approach and confirm the possibility of using affective brain-computer/machine interfaces.

[1]  Allan Kardec Barros,et al.  A Multimodal Approach to Communicative Interactivity Classification , 2007, J. VLSI Signal Process..

[2]  A. Mehrabian,et al.  A measure of emotional empathy. , 1972, Journal of personality.

[3]  Toshihisa Tanaka,et al.  Signal Processing Techniques for Knowledge Extraction and Information Fusion , 2008 .

[4]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[5]  Fionn Murtagh,et al.  Multidimensional clustering algorithms , 1985 .

[6]  Enhua Shen,et al.  Advances in Cognitive Neurodynamics ICCN 2007: Proceedings of the International Conference on Cognitive Neurodynamics. ICCN 2007 Proceedings , 2008 .

[7]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[8]  Andrzej Cichocki,et al.  Emotional States Estimation from Multichannel EEG Maps , 2008 .

[9]  Liqing Zhang,et al.  Noninvasive BCIs: Multiway Signal-Processing Array Decompositions , 2008, Computer.

[10]  Wade Junek,et al.  Mind Reading: The Interactive Guide to Emotions , 2007 .

[11]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Toshihisa Tanaka,et al.  Multichannel spectral pattern separation - An EEG processing application - , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Jonah Lehrer,et al.  How We Decide , 2009 .