Dynamic Aspects of Face Processing in Humans

In this chapter, we will focus on the role of motion in identity and expression recognition in human, and its developmental and neurophysiological aspects. Based on results from literature, we make it clear that there is some form of characteristic facial information that is only available over time, and that it plays an important role in the recognition of identity, expression, speech, and gender; and that the addition of dynamic information improves the recognizability of expressions and identity, and can compensate for the loss of static information. Moreover, at least several different types of motion seem to exist, they play different roles, and a simple rigid/nonrigid dichotomy is neither sufficient nor appropriate to describe these motions. Additional research is necessary to determine what the dynamic features for face processing are.

[1]  N. G. Hanawalt,et al.  The RÔle of the Upper and the Lower Parts of the Face as a Basis for Judging Facial Expressions: II. In Posed Expressions and “Candid-Camera” Pictures , 1944 .

[2]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[3]  K. Lander,et al.  Why are moving faces easier to recognize? , 2005, Visual Cognition.

[4]  Alan Johnston,et al.  Infants' Discrimination of Faces by Using Biological Motion Cues , 2006, Perception.

[5]  M. Herrmann,et al.  Emotions in motion: Dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations , 2009, Brain Research.

[6]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[7]  Glyn W. Humphreys,et al.  Expression is computed separately from facial identity, and it is computed separately for moving and static faces: Neuropsychological evidence , 1993, Neuropsychologia.

[8]  J. Fernández-Dols,et al.  Neutral faces in context: Their emotional meaning and their function , 1994 .

[9]  Hervé Abdi,et al.  Recognition of moving and static faces by young infants. , 2009, Child development.

[10]  Robert Stigler,et al.  Chronophotische Studien über den Umgebungskontrast , 1910, Pflüger's Archiv für die gesamte Physiologie des Menschen und der Tiere.

[11]  Christian Wallraven,et al.  Perception of Prominence Intensity in audio-visual Speech , 2007 .

[12]  A J Shinkfield,et al.  Recognition of facial emotional expressions from moving and static displays by individuals with mental retardation. , 1999, American journal of mental retardation : AJMR.

[13]  J. Schultz,et al.  Natural facial motion enhances cortical responses to faces , 2009, Experimental Brain Research.

[14]  Clinton D. Kilts,et al.  Dissociable Neural Pathways Are Involved in the Recognition of Emotion in Static and Dynamic Facial Expressions , 2003, NeuroImage.

[15]  H. Bülthoff,et al.  The inaccuracy and insincerity of real faces , 2003 .

[16]  E. Averbach,et al.  Short-term memory in vision , 1961 .

[17]  A. Pick,et al.  Intermodal perception of happy and angry expressive behaviors by seven-month-old infants. , 1992, Child development.

[18]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[19]  Roel Vertegaal.  Conversational awareness in multiparty VMC , 1997, CHI Extended Abstracts.

[20]  J. Frois-Wittman,et al.  The judgment of facial expression , 1930 .

[21]  D. Berry.  What can a moving face tell us? , 1990, Journal of personality and social psychology.

[22]  R. Borgatti,et al.  The Development of Dynamic Facial Expression Recognition at Different Intensities in 4- to 18-Year-Olds , 2010 .

[23]  John C. Tang,et al.  What video can and cannot do for collaboration: A case study , 2005, Multimedia Systems.

[24]  H. Bülthoff,et al.  Effects of temporal association on recognition memory , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[25]  H. Bülthoff,et al.  A search advantage for faces learned in motion , 2006, Experimental Brain Research.

[26]  R. Adolphs,et al.  Dissociable neural systems for recognizing emotions , 2003, Brain and Cognition.

[27]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Heinrich H. Bülthoff,et al.  Evaluation of real-world and computer-generated stylized facial expressions , 2007, TAP.

[29]  Diane J. Schiano,et al.  Communicating facial affect: it's not the realism, it's the motion , 2000, CHI Extended Abstracts.

[30]  D. Kahneman.  Method, findings, and theory in studies of visual masking. , 1968, Psychological bulletin.

[31]  J. Bavelas,et al.  "I show how you feel": Motor mimicry as a communicative act. , 1986 .

[32]  C. Wallraven,et al.  Processing of facial identity and expression: a psychophysical, physiological, and computational perspective. , 2006, Progress in brain research.

[33]  Danielle Ropar,et al.  Do the eyes have it? Inferring mental states from animated faces in autism. , 2007, Child development.

[34]  C. Tardif,et al.  Author manuscript, published in "Journal of Autism and Developmental Disorders (2007) 1469-1484" 1 , 2008 .

[35]  Alice J. O'Toole,et al.  Memory for Moving Faces: The Interplay of Two Recognition Systems , 2010 .

[36]  G. McCarthy,et al.  Dynamic perception of facial affect and identity in the human brain. , 2003, Cerebral Cortex.

[37]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[38]  Shigeru Akamatsu,et al.  Dynamic Properties Influence the Perception of Facial Expressions , 2001, Perception.

[39]  Murray White,et al.  Parts and Wholes in Expression Recognition , 2000 .

[40]  Martin A. Giese,et al.  Dynamic Faces: Insights from Experiments and Computation , 2010 .

[41]  Mark W. Becker,et al.  Volatile visual representations: Failing to detect changes in recently processed information , 2002, Psychonomic bulletin & review.

[42]  G. Pike,et al.  Recognizing moving faces: The relative contribution of motion and perspective view information. , 1997 .

[43]  Matthew Stone,et al.  Making discourse visible: coding and animating conversational facial displays , 2002, Proceedings of Computer Animation 2002 (CA 2002).

[44]  Michael T. Motley,et al.  Facial Affect and Verbal Context in Conversation: Facial Expression as Interjection. , 1993 .

[45]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[46]  A. Johnston,et al.  Categorizing sex and identity from the biological motion of faces , 2001, Current Biology.

[47]  K. Lander,et al.  Recognizing Face Identity from Natural and Morphed Smiles , 2006, Quarterly journal of experimental psychology.

[48]  Ulla Maija Valleala.  JYVÄSKYLÄ STUDIES IN EDUCATION, PSYCHOLOGY AND SOCIAL RESEARCH 280 , 2006 .

[49]  C. Deruelle,et al.  Motion and Emotion: A Novel Approach to the Study of Face Processing by Young Autistic Children , 2001, Journal of autism and developmental disorders.

[50]  A. Walker-Andrews.  Infants' perception of expressive behaviors: differentiation of multimodal information. , 1997, Psychological bulletin.

[51]  Sabine Hunnius,et al.  Gaze shifting in infancy: a longitudinal study using dynamic faces and abstract stimuli , 2004 .

[52]  Peter Bull,et al.  Body movement and emphasis in speech , 1985 .

[53]  R. Plutchik.  The emotions: Facts, theories and a new model. , 1964 .

[54]  Z. Kourtzi,et al.  A Matching Advantage for Dynamic Human Faces , 2002, Perception.

[55]  N. Troje,et al.  Range- and domain-specific exaggeration of facial speech. , 2005, Journal of vision.

[56]  A. Mehrabian,et al.  Inference of attitudes from nonverbal communication in two channels. , 1967, Journal of consulting psychology.

[57]  Dennis C. Hay,et al.  Do children utilize motion when recognizing faces? , 2008 .

[58]  Bruno G. Breitmeyer,et al.  Metacontrast with masks varying in spatial frequency and wavelength , 1991, Vision Research.

[59]  J. Cohn,et al.  Deciphering the Enigmatic Face , 2005, Psychological science.

[60]  Christopher J. Fox,et al.  Defining the face processing network: Optimization of the functional localizer in fMRI , 2009, Human brain mapping.

[61]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[62]  T. Nummenmaa.  The language of the face , 1964 .

[63]  V. Bruce,et al.  The role of movement in the recognition of famous faces , 1999, Memory & cognition.

[64]  H. Bülthoff,et al.  The use of facial motion and facial form during the processing of identity , 2003, Vision Research.

[65]  Jeffery A. Jones,et al.  Visual Prosody and Speech Intelligibility , 2004, Psychological science.

[66]  V. Bruce,et al.  The role of dynamic information in the recognition of unfamiliar faces , 1998, Memory & cognition.

[67]  Jean-Luc Schwartz,et al.  Audiovisual perception of contrastive focus in French , 2003, AVSP.

[68]  D. Massaro,et al.  Featural evaluation, integration, and judgment of facial affect. , 1997, Journal of experimental psychology. Human perception and performance.

[69]  K. Edwards,et al.  The Face of Time: Temporal Cues in Facial Expressions of Emotion , 1998 .

[70]  Paul Pauli,et al.  Electromyographic responses to static and dynamic avatar emotional facial expressions. , 2006, Psychophysiology.

[71]  J. Cohn,et al.  A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression , 2001 .

[72]  D. Simons,et al.  Failure to detect changes to people during a real-world interaction , 1998 .

[73]  Hervé Abdi,et al.  Learning the Moves: The Effect of Familiarity and Facial Motion on Person Recognition across Large Changes in Viewing Format , 2006, Perception.

[74]  Heinrich H. Bülthoff,et al.  Evaluating the perceptual realism of animated facial expressions , 2008, TAP.

[75]  J. Bavelas,et al.  Listeners as co-narrators. , 2000, Journal of personality and social psychology.

[76]  A. Young,et al.  Understanding face recognition. , 1986, British journal of psychology.

[77]  Klaus R. Scherer,et al.  Studying the dynamics of emotional expression using synthesized facial muscle movements. , 2000 .

[78]  H. Wallbott,et al.  Emotion category accessibility and the decoding of emotion from facial expression and context , 1991 .

[79]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[80]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[81]  H. Bülthoff,et al.  The contribution of different facial regions to the recognition of conversational expressions. , 2008, Journal of vision.

[82]  Helmut Leder,et al.  The development of face processing , 2003 .

[83]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[84]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[85]  A. Ishai,et al.  Distinct, overlapping representations of faces and multiple categories of objects in ventral temporal cortex , 2001, NeuroImage.

[86]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.

[87]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[88]  Mikko Sams,et al.  Identification of synthetic and natural emotional facial expressions , 2003, AVSP.

[89]  A. Johnston,et al.  The Role of Movement in Face Recognition , 1997 .

[90]  J. N. Bassili.  Facial motion in the perception of faces and of emotional expression. , 1978, Journal of experimental psychology. Human perception and performance.

[91]  C. Wallraven,et al.  Dynamic information for the recognition of conversational expressions. , 2009, Journal of vision.

[92]  V. Yngve.  On getting a word in edgewise , 1970 .

[93]  M. Moscovitch,et al.  Temporal integration in face perception: evidence of configural processing of temporally separated face parts. , 2007, Journal of experimental psychology. Human perception and performance.

[94]  N. Frijda,et al.  Dimensions of recognition of expression. , 1963, Journal of abnormal and social psychology.

[95]  Heinrich H. Bülthoff,et al.  Manipulating Video Sequences to Determine the Components of Conversational Facial Expressions , 2005, TAP.

[96]  Heinrich H. Bülthoff,et al.  How believable are real faces? Towards a perceptual basis for conversational animation , 2003, Proceedings 11th IEEE International Workshop on Program Comprehension.

[97]  D. Berry,et al.  Child and Adult Sensitivity to Gender Information in Patterns of Facial Motion , 1991 .

[98]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[99]  Silvan S. Tomkins,et al.  Affect, Cognition and Personality Empirical Studies , 1965 .

[100]  John C. Tang,et al.  What video can and cannot do for collaboration: A case study , 1993, MULTIMEDIA '93.

[101]  Alex Pentland,et al.  A vision system for observing and extracting facial action parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[102]  T. Brazelton,et al.  Monadic phases: A structural descriptive analysis of infant–mother face to face interaction. , 1980 .

[103]  Hao Yan,et al.  More than just a pretty face: conversational protocols and the affordances of embodiment , 2001, Knowl. Based Syst..

[104]  J. Bavelas,et al.  Visible Acts of Meaning , 2000 .

[105]  Jesper Kjeldskov,et al.  Using mobile phones to support sustainability: a field study of residential electricity consumption , 2012, CHI.

[106]  J. Barton,et al.  Disorders of face perception and recognition. , 2003, Neurologic clinics.

[107]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[108]  Vicki Bruce,et al.  Recognizing Famous Faces: Exploring the Benefits of Facial Motion , 2000 .

[109]  Vicki Bruce,et al.  Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces , 2001 .

[110]  W. S. Condon,et al.  SOUND FILM ANALYSIS OF NORMAL AND PATHOLOGICAL BEHAVIOR PATTERNS , 1966, The Journal of nervous and mental disease.