HapFACS: An Open Source API/Software to Generate FACS-Based Expressions for ECAs Animation and for Corpus Generation

We present HapFACS (ver. beta), a new open source software and API for generating FACS-based facial expressions on 3D virtual characters that have accompanying lip-synchronized animation abilities. HapFACS has two main usage scenarios: First, with the HapFACS software, users can generate repertoires of realistic FACS-validated facial expressions, either as static images or as videos, Second, with the accessible HapFACS API, users can animate speaking virtual characters with real-time realistic facial expressions, and embed these expressive characters in their own application(s) without any prior experience in computer graphics and modeling. We describe how HapFACS (1) provides control over 49 FACS Action Units at all levels of intensity, (2) enables the animation of faces with a single AU or a composition of AUs, activated unilaterally or bilaterally, and (3) can be applied to any supported character in the underlying 3D-character system1. Finally, we provide details of evaluation experiments we conducted with FACS-certified scorers to validate the facial expressions generated by HapFACS.

[1]  Anton Leuski,et al.  Virtual Patients for Clinical Therapist Skills Training , 2007, IVA.

[2]  K. Shockley,et al.  Using emotional cues in a discrimination learning task: Effects of trait emotional intelligence and affective state , 2012 .

[3]  Steve DiPaola,et al.  Socially Communicative Characters for Interactive Applications , 2006 .

[4]  K. Scherer,et al.  Appraisal processes in emotion: Theory, methods, research. , 2001 .

[5]  Tibor Bosse,et al.  An Intelligent Virtual Agent to Increase Involvement in Financial Services , 2010, IVA.

[6]  Skyler T. Hawk,et al.  Moving faces, looking places: validation of the Amsterdam Dynamic Facial Expression Set (ADFES). , 2011, Emotion.

[7]  A. J. Fridlund.  Human Facial Expression: An Evolutionary View , 1994 .

[8]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[9]  Stacy Marsella,et al.  Expressive Behaviors for Virtual Worlds , 2004, Life-like characters.

[10]  Eva G Krumhuber,et al.  FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research. , 2012, Emotion.

[11]  J. Allwood.  BODILY COMMUNICATION DIMENSIONS OF EXPRESSION AND CONTENT , 2002 .

[12]  Nicole Chovil.  Discourse‐oriented facial displays in conversation , 1991 .

[13]  Chao Fan,et al.  See Me, Teach Me: Facial Expression and Gesture Recognition for Intelligent Tutoring Systems , 2006, 2006 Innovations in Information Technology.

[14]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[15]  J. E. Rose,et al.  Autonomic Nervous System Activity Distinguishes Among Emotions , 2009 .

[16]  D. Lundqvist,et al.  Karolinska Directed Emotional Faces , 2015 .

[17]  Ari Shapiro,et al.  Building a Character Animation System , 2011, MIG.

[18]  Florian Schiel,et al.  The SmartKom Multimodal Corpus at BAS , 2002, LREC.

[19]  P. Ekman,et al.  Detecting deception from the body or face. , 1974 .

[20]  J. Cohn,et al.  A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression , 2001 .

[21]  P. Ekman,et al.  EMFACS-7: Emotional Facial Action Coding System , 1983 .

[22]  Katherine B. Martin,et al.  Facial Action Coding System , 2015 .

[23]  Marc Cavazza,et al.  Multimodal and mobile conversational Health and Fitness Companions , 2011, Comput. Speech Lang..

[24]  Naphtali Rishe,et al.  Building an On-Demand Avatar-Based Health Intervention for Behavior Change , 2012, FLAIRS.

[25]  Sergi Villagrasa,et al.  FACe! 3D Facial Animation System based on FACS , 2009 .

[26]  Franco Casalino,et al.  MPEG-4: A Multimedia Standard for the Third Millennium, Part 1 , 1999, IEEE Multim..

[27]  Jessica L. Tracy,et al.  Development of a Facs-verified Set of Basic and Self-conscious Emotion Expressions Extant Facs-verified Sets , 2009 .

[28]  Julian Szymanski,et al.  Information retrieval with semantic memory model , 2012, Cognitive Systems Research.

[29]  Leon Beutl.  A simulation for the creation of soft-looking, realistic facial expressions , 2011 .