Implicit affective profiling of subjects based on physiological data coupling

Evidence from neuroscience and psychology supports the idea that synchronization among various sub-components of the organism is the key-point to emotional processes. This study investigates the way signals from the central nervous system interact with signals from the peripheral nervous system during emotional processes elicited by music clips. In particular, the study reveals that coupling between skin conductance response (SCR) and electroencephalography (EEG) of the temporal lobe is significantly stronger compared to the coupling between SCR and EEG captured from other parts of the cortex (p < 0.01) during music clips. It is also shown that the coupling between SCR and EEG of the temporal lobe increases significantly during high arousing music clips with respect to low arousing ones ( p < 0.01), as well as during highly pleasant ( p < 0.01) or very unpleasant ( p < 0.01) music clips with respect to neutral ones. The potential of including such coupling between EEG and SCR in an affective brain-compute...

[1]  R. Dolan,et al.  A Functional Anatomy of Anticipatory Anxiety , 1999, NeuroImage.

[2]  Enzo Pasquale Scilingo,et al.  Oscillations of Heart Rate and Respiration Synchronize During Affective Visual Stimulation , 2012, IEEE Transactions on Information Technology in Biomedicine.

[3]  K. Scherer,et al.  Conscious emotional experience emerges as a function of multilevel, appraisal-driven response synchronization , 2008, Consciousness and Cognition.

[4]  R. Davidson.  What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research , 2004, Biological Psychology.

[5]  Heidi M. Rockwood,et al.  Huygens's clocks , 2002, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[6]  R. J Dolan,et al.  Activity in ventromedial prefrontal cortex covaries with sympathetic skin conductance level: a physiological account of a “default mode” of brain function , 2004, NeuroImage.

[7]  G. Pfurtscheller,et al.  Correlation between EEG burst-to-burst intervals and HR acceleration in preterm infants , 2008, Neuroscience Letters.

[8]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[9]  Karl J. Friston,et al.  A neuromodulatory role for the human amygdala in processing emotional facial expressions. , 1998, Brain : a journal of neurology.

[10]  S. Stouffer,et al.  The American soldier : combat and its aftermath , 1949 .

[11]  A. Hochschild.  Emotion Work, Feeling Rules, and Social Structure , 1979, American Journal of Sociology.

[12]  J. Buck.  Synchronous Rhythmic Flashing of Fireflies. II. , 1938, The Quarterly Review of Biology.

[13]  David Sander,et al.  A systems approach to appraisal mechanisms in emotion , 2005, Neural Networks.

[14]  Rosalind W. Picard.  Affective Computing , 1997 .

[15]  R. Tibshirani,et al.  An Introduction to the Bootstrap , 1995 .

[16]  Alicia A. Grandey,et al.  Emotion regulation in the workplace: a new way to conceptualize emotional labor. , 2000, Journal of occupational health psychology.

[17]  A. A. Lumsdaine,et al.  The American soldier: Combat and its aftermath. (Studies in social psychology in World War II), Vol. 2 , 1949 .

[18]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[19]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[20]  Hatice Gunes,et al.  From the Lab to the real world: affect recognition using multiple cues and modalities , 2008 .

[21]  Touradj Ebrahimi,et al.  Multivariate spectral analysis for identifying the brain activations during olfactory perception , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  M. Rosenblum,et al.  Chapter 9 Phase synchronization: From theory to data analysis , 2001 .

[23]  Hugo D. Critchley,et al.  Volitional Control of Autonomic Arousal: A Functional Magnetic Resonance Study , 2002, NeuroImage.

[24]  Mohammad Soleymani,et al.  Single Trial Classification of EEG and Peripheral Physiological Signals for Recognition of Emotions Induced by Music Videos , 2010, Brain Informatics.

[25]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Karl J. Friston,et al.  Experience–dependent modulation of tonotopic neural responses in human auditory cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[27]  A. Opstal.  Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[28]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[29]  D. Gabor,et al.  Theory of communication. Part 1: The analysis of information , 1946 .

[30]  C. Simon,et al.  Inverse coupling between ultradian oscillations in delta wave activity and heart rate variability during sleep , 2001, Clinical Neurophysiology.

[31]  Touradj Ebrahimi,et al.  Electroencephalogram alterations during perception of pleasant and unpleasant odors , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[32]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[33]  T. Hendler,et al.  Feeling or Features Different Sensitivity to Emotion in High-Order Visual Cortex and Amygdala , 2001, Neuron.

[34]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[35]  P. Ekman.  An argument for basic emotions , 1992 .

[36]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[37]  Maja Pantic,et al.  Implicit human-centered tagging [Social Sciences] , 2009, IEEE Signal Process. Mag..

[38]  M. McClintock,et al.  Menstrual Synchrony and Suppression , 1971, Nature.

[39]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[40]  Björn W. Schuller,et al.  Categorical and dimensional affect analysis in continuous input: Current trends and future directions , 2013, Image Vis. Comput..

[41]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[42]  Touradj Ebrahimi,et al.  EEG Correlates of Different Emotional States Elicited during Watching Music Videos , 2011, ACII.

[43]  O. John,et al.  Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. , 2003, Journal of personality and social psychology.

[44]  H. Fujisaka,et al.  Stability Theory of Synchronized Motion in Coupled-Oscillator Systems , 1983 .

[45]  K. Scherer.  Neuroscience projections to current debates in emotion psychology , 1993 .

[46]  M. Lambertz,et al.  Simultaneous changes of rhythmic organization in brainstem neurons, respiration, cardiovascular system and EEG between 0.05 Hz and 0.5 Hz. , 1998, Journal of the autonomic nervous system.

[47]  C. Spearman.  The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[48]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[49]  J. Russell,et al.  A cross-cultural study of a circumplex model of affect. , 1989 .

[50]  A. Damasio.  The somatic marker hypothesis and the possible functions of the prefrontal cortex. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[51]  H H Abel,et al.  Synchronization in the human cardiorespiratory system. , 1998, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[52]  M. Bradley,et al.  Looking at pictures: affective, facial, visceral, and behavioral reactions. , 1993, Psychophysiology.

[53]  J. Kurths,et al.  Phase synchronization: from theory to data analysis , 2003 .

[54]  R. Dolan,et al.  Emotion, Decision Making, and the Amygdala , 2008, Neuron.

[55]  M. Dumont,et al.  Interdependency between heart rate variability and sleep EEG: linear/non-linear? , 2004, Clinical Neurophysiology.

[56]  Mohammad Soleymani,et al.  Human-centered implicit tagging: Overview and perspectives , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[57]  B. Sinha,et al.  Statistical Meta-Analysis with Applications , 2008 .

[58]  A. Damasio.  Descartes' error: emotion, reason, and the human brain. avon books , 1994 .

[59]  D. Kumaran,et al.  Frames, Biases, and Rational Decision-Making in the Human Brain , 2006, Science.

[60]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[61]  Enzo Pasquale Scilingo,et al.  The Role of Nonlinear Dynamics in Affective Valence and Arousal Recognition , 2012, IEEE Transactions on Affective Computing.

[62]  Leontios J. Hadjileontiadis,et al.  Toward an EEG-Based Recognition of Music Liking Using Time-Frequency Analysis , 2012, IEEE Transactions on Biomedical Engineering.

[63]  H. Eichenbaum,et al.  Measuring phase-amplitude coupling between neuronal oscillations of different frequencies. , 2010, Journal of neurophysiology.

[64]  Sylvia D. Kreibig,et al.  An affective computing approach to physiological emotion specificity: toward subject-independent and stimulus-independent classification of film-induced emotions. , 2011, Psychophysiology.

[65]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[66]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[67]  Ian J. Deary,et al.  Cognitive ability and simple reaction time predict cardiac reactivity in the West of Scotland Twenty-07 Study: Cognition and reactivity , 2011 .

[68]  M. Whitlock.  Combining probability from independent tests: the weighted Z‐method is superior to Fisher's approach , 2005, Journal of evolutionary biology.

[69]  Jennifer A. Healey,et al.  Wearable and automotive systems for affect recognition from physiology , 2000 .

[70]  J. Andersson,et al.  Functional neuroanatomical correlates of electrodermal activity: a positron emission tomographic study. , 1998, Psychophysiology.

[71]  G. Pfurtscheller,et al.  Coupling between Intrinsic Prefrontal HbO2 and Central EEG Beta Power Oscillations in the Resting Brain , 2012, PloS one.

[72]  Joseph E LeDoux.  The Emotional Brain: The Mysterious Underpinnings of Emotional Life , 1996 .

[73]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[74]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[75]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[76]  G. Loewenstein,et al.  The role of affect in decision making. , 2003 .

[77]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .