Recognition of Intensive Valence and Arousal Affective States via Facial Electromyographic Activity in Young and Senior Adults

Background Research suggests that interaction between humans and digital environments characterizes a form of companionship in addition to technical convenience. To this effect, humans have attempted to design computer systems able to demonstrably empathize with the human affective experience. Facial electromyography (EMG) is one such technique enabling machines to access to human affective states. Numerous studies have investigated the effects of valence emotions on facial EMG activity captured over the corrugator supercilii (frowning muscle) and zygomaticus major (smiling muscle). The arousal emotion, specifically, has not received much research attention, however. In the present study, we sought to identify intensive valence and arousal affective states via facial EMG activity. Methods Ten blocks of affective pictures were separated into five categories: neutral valence/low arousal (0VLA), positive valence/high arousal (PVHA), negative valence/high arousal (NVHA), positive valence/low arousal (PVLA), and negative valence/low arousal (NVLA), and the ability of each to elicit corresponding valence and arousal affective states was investigated at length. One hundred and thirteen participants were subjected to these stimuli and provided facial EMG. A set of 16 features based on the amplitude, frequency, predictability, and variability of signals was defined and classified using a support vector machine (SVM). Results We observed highly accurate classification rates based on the combined corrugator and zygomaticus EMG, ranging from 75.69% to 100.00% for the baseline and five affective states (0VLA, PVHA, PVLA, NVHA, and NVLA) in all individuals. There were significant differences in classification rate accuracy between senior and young adults, but there was no significant difference between female and male participants. Conclusion Our research provides robust evidences for recognition of intensive valence and arousal affective states in young and senior adults. These findings contribute to the successful future application of facial EMG for identifying user affective states in human machine interaction (HMI) or companion robotic systems (CRS).

[1]  A. J. Fridlund,et al.  Guidelines for human electromyographic research. , 1986, Psychophysiology.

[2]  Jing Zhang,et al.  The effects of arousal and valence on facial electromyographic asymmetry during blocked picture viewing. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  David J. Klein,et al.  Psychophysiology of Emotion Across the Life Span , 1997, Annual Review of Gerontology and Geriatrics.

[4]  Ganesh R. Naik,et al.  Computational Intelligence in Electromyography Analysis - A Perspective on Current Applications and Future Challenges , 2012 .

[5]  M. Guha.  The Oxford Companion to Emotion and the Affective Sciences , 2010 .

[6]  J. Cacioppo,et al.  The psychophysiology of emotion. , 1993 .

[7]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[8]  J. Cacioppo,et al.  Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. , 1986, Journal of personality and social psychology.

[9]  L. Carstensen,et al.  Emotional experience in everyday life across the adult life span. , 2000, Journal of personality and social psychology.

[10]  James J. Gross,et al.  Emotion Generation and Emotion Regulation: One or Two Depends on Your Point of View , 2011, Emotion review : journal of the International Society for Research on Emotion.

[11]  Slawomir J. Nasuto,et al.  EMG Decomposition and Artefact Removal , 2012 .

[12]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999, Journal of personality and social psychology.

[13]  Laura L. Carstensen,et al.  At the Intersection of Emotion and Cognition , 2005 .

[14]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999 .

[15]  B. Englis,et al.  Expectations of Cooperation and Competition and Their Effects on Observers' Vicarious Emotional Responses , 1989 .

[16]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[17]  Catherine J. Norris,et al.  Aging is associated with positive responding to neutral information but reduced recovery from negative information. , 2011, Social cognitive and affective neuroscience.

[18]  Friedhelm Schwenker,et al.  Multimodal Emotion Classification in Naturalistic User Behavior , 2011, HCI.

[19]  J. Russell,et al.  An approach to environmental psychology , 1974 .

[20]  M. Bradley,et al.  Looking at pictures: affective, facial, visceral, and behavioral reactions. , 1993, Psychophysiology.

[21]  Cynthia A Berg,et al.  Are older adults less or more physiologically reactive? A meta-analysis of age-related differences in cardiovascular reactivity to laboratory tasks. , 2010, The journals of gerontology. Series B, Psychological sciences and social sciences.

[22]  Alfred W. Kaszniak,et al.  Age-Invariance in the Asymmetry of Stimulus-Evoked Emotional Facial Muscle Activity , 2000 .

[23]  Stefanie Rukavina,et al.  Expression intensity, gender and facial emotion recognition: Women recognize only subtle facial emotions better than men. , 2010, Acta psychologica.

[24]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[25]  J. Larsen,et al.  A facial electromyographic investigation of affective contrast. , 2009, Psychophysiology.

[26]  C. V. Witvliet,et al.  Psychophysiological responses as indices of affective dimensions. , 1995, Psychophysiology.

[27]  Lindsey A. Leigland,et al.  Age differences in perception and awareness of emotion , 2009, Neurobiology of Aging.

[28]  J. Henry,et al.  A meta-analytic review of emotion recognition and aging: Implications for neuropsychological models of aging , 2008, Neuroscience & Biobehavioral Reviews.

[29]  Jeff T. Larsen,et al.  Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. , 2003, Psychophysiology.

[30]  Andreas Wendemuth,et al.  Companion-Technology for Cognitive Technical Systems , 2016, KI - Künstliche Intelligenz.

[31]  Alberto Radaelli,et al.  Invited review: aging and the cardiovascular system. , 2003, Journal of applied physiology.

[32]  A. Mehrabian.  Framework for a comprehensive description and measurement of emotional states. , 1995, Genetic, social, and general psychology monographs.

[33]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[34]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[35]  Peter J. Kyberd,et al.  Bridging the gap between robotic technology and health care , 2014, Biomed. Signal Process. Control..

[36]  Orlando Francisco Amodeo Bueno,et al.  IAPS includes photographs that elicit low-arousal physiological responses in healthy volunteers , 2007, Physiology & Behavior.

[37]  Holger Hoffmann,et al.  Repeatability of facial electromyography (EMG) activity over corrugator supercilii and zygomaticus major on differentiating various emotions , 2011, Journal of Ambient Intelligence and Humanized Computing.

[38]  Anton van Boxtel,et al.  Facial EMG as a tool for inferring affective states , 2010 .

[39]  R. Larsen,et al.  Promises and problems with the circumplex model of emotion. , 1992 .

[40]  C. Bulpitt,et al.  Aging and the cardiovascular system. , 1978, Acta cardiologica.

[41]  Human-Computer Interaction. Towards Mobile and Intelligent Interaction Environments , 2011, Lecture Notes in Computer Science.

[42]  L. Carstensen,et al.  Autonomic, subjective, and expressive responses to emotional films in older and younger Chinese Americans and European Americans. , 2000, Psychology and aging.

[43]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[44]  M. Lawton,et al.  Focus on emotion and adult development , 1998 .

[45]  H. Traue,et al.  Pain Intensity Recognition Rates via Biopotential Feature Patterns with Support Vector Machines , 2015, PloS one.

[46]  Susanne Scheibe,et al.  Emotional aging: recent findings and future trends. , 2010, The journals of gerontology. Series B, Psychological sciences and social sciences.

[47]  D. Tsiachris,et al.  Aging and the cardiovascular system. , 2010, Hellenic journal of cardiology : HJC = Hellenike kardiologike epitheorese.

[48]  Charles H Hillman,et al.  Influences of age on emotional reactivity during picture processing. , 2005, The journals of gerontology. Series B, Psychological sciences and social sciences.

[49]  Jonghwa Kim,et al.  Transsituational Individual-Specific Biopsychological Classification of Emotions , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[50]  Bernhard Dahme,et al.  Gradients of Facial EMG and Cardiac Activity During Emotional Stimulation , 1999 .

[51]  Yorick Wilks.  Introducing artificial Companions , 2010 .

[52]  Harald C. Traue,et al.  The influence of neuroticism and psychological symptoms on the assessment of images in three-dimensional emotion space , 2011, Psycho-social medicine.

[53]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[54]  J. Broekens,et al.  Assistive social robots in elderly care: a review , 2009 .

[55]  Friedhelm Schwenker,et al.  A Framework for Emotions and Dispositions in Man-Companion Interaction , 2013 .

[56]  D. Powell,et al.  Psychophysiological and subjective indices of emotion as a function of age and gender , 2007 .

[57]  Othman Omran Khalifa,et al.  EMG signal classification for human computer interaction: a review , 2009 .