A Natural Conversational Virtual Human with Multimodal Dialog System

The making of virtual human character to be realistic and credible in real time automated dialog animation system is necessary. This kind of animation carries importance elements for many applications such as games, virtual agents and movie animations. It is also considered important for applications which require interaction between human and computer. However, for this purpose, it is compulsory that the machine should have sufficient intelligence for recognizing and synthesizing human voices. As one of the most vital interaction method between human and machine, speech has recently received significant attention, especially in avatar research innovation. One of the challenges is to create precise lip movements of the avatar and synchronize it with a recorded audio. This paper specifically introduces the innovative concept of multimodal dialog systems of the virtual character and focuses the output part of such systems. More specifically, its focus is on behavior planning and developing the data control languages (DCL).

[1]  Gabriel Skantze,et al.  IrisTK: a statechart-based toolkit for multi-party face-to-face interaction , 2012, ICMI '12.

[2]  Catherine Pelachaud,et al.  Expressive Gesture Model for Humanoid Robot , 2011, ACII.

[3]  Justine Cassell,et al.  Human conversation as a system framework: designing embodied conversational agents , 2001 .

[4]  Ken-ichi Anjyo,et al.  Developing tools for 2D/3D conversion of Japanese animations , 2011, SIGGRAPH '11.

[5]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.

[6]  Igor S. Pandzic,et al.  Multimodal behavior realization for embodied conversational agents , 2011, Multimedia Tools and Applications.

[7]  David R. Traum,et al.  The BML Sequencer: A Tool for Authoring Multi-character Animations , 2011, IVA.

[8]  Gaëlle Ferré,et al.  Timing Relationships between Speech and Co-Verbal Gestures in Spontaneous French , 2010, LREC 2010.

[9]  Wolfgang Wahlster,et al.  Dialogue Systems Go Multimodal: The SmartKom Experience , 2006, SmartKom.

[10]  Xiaopeng Wei,et al.  Facial Animation Based on Feature Points , 2013 .

[11]  Louis-Philippe Morency,et al.  Cicero - Towards a Multimodal Virtual Audience Platform for Public Speaking Training , 2013, IVA.

[12]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[13]  Igor S. Pandzic,et al.  On creating multimodal virtual humans—real time speech driven facial gesturing , 2010, Multimedia Tools and Applications.

[14]  José Colás Pasamontes,et al.  Spanish Sign Language synthesis system , 2012, J. Vis. Lang. Comput..

[15]  Yonghong Yan,et al.  Universal speech tools: the CSLU toolkit , 1998, ICSLP.

[16]  Daniel Thalmann,et al.  Believable Virtual Characters in Human-Computer Dialogs , 2011, Eurographics.

[17]  Jan Kleindienst,et al.  ECAF: Authoring Language for Embodied Conversational Agents , 2007, TSD.