Compound Gesture Generation: A Model Based on Ideational Units

This work presents a hierarchical framework that generates continuous gesture animation performance for virtual characters. As opposed to approaches that focus more on realizing individual gesture, the focus of this work is on the relation between gestures as part of an overall gesture performance. Following Calbris’ work [3], our approach is to structure the performance around ideational units and determine gestural features within and across these ideational units. Furthermore, we use Calbris’ work on the relation between form and meaning in gesture to help inform how individual gesture’s expressivity is manipulated. Our framework takes in high level communicative function descriptions, generates behavior descriptions and realizes them using our character animation engine. We define the specifications for these different levels of descriptions. Finally, we show the general results as well as experiments illustrating the impacts of the key features.

[1]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[2]  Stacy Marsella,et al.  Gesture with Meaning , 2013, IVA.

[3]  Yuyu Xu,et al.  Virtual character performance from speech , 2013, SCA '13.

[4]  M. Studdert-Kennedy Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[5]  Michael Neff,et al.  Towards Natural Gesture Synthesis: Evaluating Gesture Units in a Data-Driven Approach to Gesture Synthesis , 2007, IVA.

[6]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.

[7]  A. Kendon Some Relationships Between Body Motion and Speech , 1972 .

[8]  Radoslaw Niewiadomski,et al.  Greta: an interactive expressive ECA system , 2009, AAMAS.

[9]  Hans-Peter Seidel,et al.  Annotated New Text Engine Animation Animation Lexicon Animation Gesture Profiles MR : . . . JL : . . . Gesture Generation Video Annotated Gesture Script , 2007 .

[10]  John Hart,et al.  ACM Transactions on Graphics: Editorial , 2003, SIGGRAPH 2003.

[11]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[12]  Geneviève Calbris,et al.  Elements of Meaning in Gesture , 2011 .

[13]  A. Kendon Gesture: Visible Action as Utterance , 2004 .

[14]  Stacy Marsella,et al.  Nonverbal Behavior Generator for Embodied Conversational Agents , 2006, IVA.

[15]  Judith A. Hall,et al.  Nonverbal behavior in clinician—patient interaction , 1995 .

[16]  D. McNeill Gesture and Thought , 2005 .

[17]  J. D. Ruiter The production of gesture and speech , 2000 .

[18]  Yuyu Xu,et al.  An example-based motion synthesis technique for locomotion and object manipulation , 2012, I3D '12.

[19]  Stacy Marsella,et al.  SmartBody: behavior realization for embodied conversational agents , 2008, AAMAS.

[20]  Ipke Wachsmuth,et al.  Max - A Multimodal Assistant in Virtual Reality Construction , 2003, Künstliche Intell..

[21]  Carolyn G. Fidelman,et al.  The semiotics of French gestures , 1990 .