Head X: Customizable Audiovisual Synthesis for a Multi-purpose Virtual Head

The development of embodied conversational agents (ECAs) involves a wide range of cutting-edge technologies extending from multimodal perception to reasoning to synthesis. While each is important to a successful outcome, it is the synthesis that has the most immediate impact on the observer. The specific appearance and voice of an embodied conversational agent (ECA) can be decisive factors in meeting its social objectives. In light of this, we have developed an extensively customizable system for synthesizing a virtual talking 3D head. Rather than requiring explicit integration into a codebase, our software runs as a service that can be controlled by any external client, which substantially simplifies its deployment into new applications. We have explored the benefits of this approach across several internal research projects and student exercises as part of a university topic on ECAs.

[1]  Vincent G. Duffy.  Digital Human Modeling , 2009, Lecture Notes in Computer Science.

[2]  R. Moreno,et al.  Students' choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity , 2006 .

[3]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[4]  David M. W. Powers,et al.  MANA for the Ageing , 2010 .

[5]  Antonio Camurri,et al.  Gesture-Based Communication in Human-Computer Interaction , 2003, Lecture Notes in Computer Science.

[6]  J. V. Kuppevelt,et al.  Advances in natural multimodal dialogue systems , 2005 .

[7]  Justine Cassell,et al.  Embodied Conversational Agents: Representation and Intelligence in User Interfaces , 2001, AI Mag..

[8]  C. Pelachaud,et al.  GRETA. A BELIEVABLE EMBODIED CONVERSATIONAL AGENT , 2005 .

[9]  Trent W. Lewis,et al.  Development of a virtual agent based social tutor for children with autism spectrum disorders , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[10]  Matthew Stone,et al.  Understanding RUTH: Creating Believable Behaviors for a Virtual Human Under Uncertainty , 2007, HCI.

[11]  J. Cassell,et al.  Social Dialongue with Embodied Conversational Agents , 2005 .

[12]  Alice Wang,et al.  Assembling an expressive facial animation system , 2007, Sandbox '07.

[13]  J. Cassell,et al.  SOCIAL DIALOGUE WITH EMBODIED CONVERSATIONAL AGENTS , 2005 .

[14]  Oliviero Stock,et al.  Multimodal intelligent information presentation , 2005 .

[15]  Marc Schröder,et al.  The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching , 2003, Int. J. Speech Technol..

[16]  D. Massaro.  From Multisensory Integration to Talking Heads and Language Learning , 2002 .

[17]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[18]  Magnus Haake,et al.  Design of animated pedagogical agents - A look at their look , 2006, Int. J. Hum. Comput. Stud..

[19]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[20]  Stefan Kopp,et al.  Imitation Games with an Artificial Agent: From Mimicking to Understanding Shape-Related Iconic Gestures , 2003, Gesture Workshop.