Walk the talk: coordinating gesture with locomotion for conversational characters

Communicative behaviors are a very important aspect of human behavior and deserve special attention when simulating groups and crowds of virtual pedestrians. Previous approaches have tended to focus on generating believable gestures for individual characters and talker‐listener behaviors for static groups. In this paper, we consider the problem of creating rich and varied conversational behaviors for data‐driven animation of walking and jogging characters. We captured ground truth data of participants conversing in pairs while walking and jogging. Our stylized splicing method takes as input a motion captured standing gesture performance and a set of looped full body locomotion clips. Guided by the ground truth metrics, we perform stylized splicing and synchronization of gesture with locomotion to produce natural conversations of characters in motion. Copyright © 2016 John Wiley & Sons, Ltd.

[1]  Lap-Pui Chau,et al.  Synchronized partial-body motion graphs , 2010, SIGGRAPH ASIA.

[2]  Stacy Marsella,et al.  Expressive Behaviors for Virtual Worlds , 2004, Life-like characters.

[3]  C. O'Sullivan,et al.  Seeing is believing: body motion dominates in multisensory conversations , 2010, ACM Trans. Graph..

[4]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH Classes.

[5]  Judith A. Hall,et al.  Beliefs about female and male nonverbal communication , 1995 .

[6]  Jessica K. Hodgins,et al.  Construction and optimal search of interpolated motion graphs , 2007, ACM Trans. Graph..

[7]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents: Research Articles , 2004 .

[8]  Michael Neff,et al.  A Perceptual Study of the Relationship between Posture and Gesture for Virtual Characters , 2012, MIG.

[9]  Sergey Levine,et al.  Gesture controllers , 2010, SIGGRAPH 2010.

[10]  J. P. Foley,et al.  Gesture and Environment , 1942 .

[11]  Sergey Levine,et al.  Real-time prosody-driven synthesis of body language , 2009, ACM Trans. Graph..

[12]  Hans-Peter Seidel,et al.  Annotated New Text Engine Animation Animation Lexicon Animation Gesture Profiles MR : . . . JL : . . . Gesture Generation Video Annotated Gesture Script , 2007 .

[13]  G Scott.  Walk the talk. , 2000, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[14]  Carol O'Sullivan,et al.  Seeing is believing: body motion dominates in multisensory conversations , 2010, SIGGRAPH 2010.

[15]  Victor B. Zordan,et al.  Automatic splicing for hand and body animations , 2006, SCA '06.

[16]  Sotaro Kita,et al.  Movement Phase in Signs and Co-Speech Gestures, and Their Transcriptions by Human Coders , 1997, Gesture Workshop.

[17]  D. McNeill.  Gesture and Thought , 2005 .

[18]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[19]  Michael Neff,et al.  The Influence of Prosody on the Requirements for Gesture-Text Alignment , 2013, IVA.

[20]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[21]  Hyun Joon Shin,et al.  Snap-together motion: assembling run-time animations , 2003, SIGGRAPH '08.

[22]  Sergey Levine,et al.  Real-time prosody-driven synthesis of body language , 2009, SIGGRAPH 2009.

[23]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[24]  Jan Peter De Ruiter,et al.  On the audiovisual integration of speech and gesture , 2012 .

[25]  Matthew Stone,et al.  Speaking with hands: creating animated conversational characters from recordings of human performance , 2004, ACM Trans. Graph..

[26]  A. Fernández-Baena,et al.  BodySpeech : A configurable facial and gesture animation system for speaking avatars , 2013 .

[27]  Ludovic Hoyet,et al.  Evaluating the distinctiveness and attractiveness of human motions on realistic virtual bodies , 2013, ACM Trans. Graph..

[28]  Lucas Kovar,et al.  Splicing Upper‐Body Actions with Locomotion , 2006, Comput. Graph. Forum.

[29]  James K. Hahn,et al.  Combined Partial Motion Clips , 2003, WSCG.

[30]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[31]  Christos Mousas,et al.  Splicing of Concurrent Upper-body Motion Spaces with Locomotion , 2013, VARE.

[32]  Eric Horvitz,et al.  Facilitating multiparty dialog with gaze, gesture, and speech , 2010, ICMI-MLMI '10.