The semantic space for facial communication

We can learn a lot about someone by watching their facial expressions and body language. Harnessing these aspects of non‐verbal communication can lend artificial communication agents greater depth and realism but requires a sound understanding of the relationship between cognition and expressive behaviour. Here, we extend traditional word‐based methodology to use actual videos and then extract the semantic/cognitive space of facial expressions. We find that depending on the specific expressions used, either a four‐dimensional or a two‐dimensional space is needed to describe the variance in the stimuli. The shape and structure of the 4D and 2D spaces are related to each other and very stable to methodological changes. The results show that there is considerable variance between how different people express the same emotion. The recovered space can well capture the full range of facial communication and is very suitable for semantic‐driven facial animation. Copyright © 2014 John Wiley & Sons, Ltd.

[1]  C. Wallraven,et al.  Dynamic information for the recognition of conversational expressions. , 2009, Journal of vision.

[2]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[3]  E. N. Sokolov,et al.  A psychophysiological model of emotion space , 2000, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[4]  H. Wallbott,et al.  Emotion category accessibility and the decoding of emotion from facial expression and context , 1991 .

[5]  A. Mehrabian,et al.  Inference of attitudes from nonverbal communication in two channels. , 1967, Journal of consulting psychology.

[6]  Christian Wallraven,et al.  Experimental Design: From User Studies to Psychophysics , 2011 .

[7]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[8]  Louis Vuurpijl,et al.  SLOT: A research platform for investigating multimodal communication , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[9]  Norman I. Badler,et al.  Authoring Multi-actor Behaviors in Crowds with Diverse Personalities , 2013, Modeling, Simulation and Visual Analysis of Crowds.

[10]  J. M. Digman PERSONALITY STRUCTURE: EMERGENCE OF THE FIVE-FACTOR MODEL , 1990 .

[11]  V. Yngve On getting a word in edgewise , 1970 .

[12]  James A. Russell,et al.  Structure of Self-Reported Current Affect : Integration and Beyond , 2004 .

[13]  Heinrich H. Bülthoff,et al.  Manipulating Video Sequences to Determine the Components of Conversational Facial Expressions , 2005, TAP.

[14]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[15]  J. Russell A circumplex model of affect. , 1980 .

[16]  H. Bülthoff,et al.  The MPI Facial Expression Database — A Validated Database of Emotional and Conversational Facial Expressions , 2012, PloS one.

[17]  J. Fernández-Dols,et al.  Neutral faces in context: Their emotional meaning and their function , 1994 .

[18]  Daniel Thalmann,et al.  Asymmetrical Facial Expressions based on an Advanced Interpretation of Two-dimensional Russells Emotional Model , 2010 .

[19]  John Funge,et al.  Cognitive modeling: knowledge, reasoning and planning for intelligent characters , 1999, SIGGRAPH.

[20]  Emiel Krahmer,et al.  Pitch, eyebrows and the perception of focus , 2002, Speech Prosody 2002.

[21]  Huailin Dong,et al.  Research on emotion modeling based on three-dimension emotional space , 2012 .

[22]  Dane Archer,et al.  Words and everything else: Verbal and nonverbal cues in social interpretation. , 1977 .

[23]  J. M. Kittross The measurement of meaning , 1959 .

[24]  Norman I. Badler,et al.  The EMOTE model for effort and shape , 2000, SIGGRAPH.

[25]  Jovan Popovic,et al.  Style translation for human motion , 2005, ACM Trans. Graph..

[26]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[27]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[28]  L. J. Williams,et al.  Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives , 1997 .

[29]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[30]  N. Badler,et al.  Toward Representing Agent Behaviors Modified by Personality and Emotion , 2002 .

[31]  W. S. Condon,et al.  SOUND FILM ANALYSIS OF NORMAL AND PATHOLOGICAL BEHAVIOR PATTERNS , 1966, The Journal of nervous and mental disease.

[32]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[33]  R. MacCallum,et al.  THE APPLICATION OF EXPLORATORY FACTOR ANALYSIS IN APPLIED PSYCHOLOGY: A CRITICAL REVIEW AND ANALYSIS , 1986 .