A Common Gesture and Speech Production Framework for Virtual and Physical Agents

We introduce a modular system to generate communicative expressive gestures accompanying speech for an agent. This system is designed as a common model for different embodiments so that its processes are independent from a specific agent. There are two main features of this system. Firstly gesture expressivity is taken into account when gesture animation are computed on the fly from abstract gesture templates. Secondly gestures are scheduled to ensure that their execution are tightly tied to speech. In this paper, we present the first implementation of this system being used to control co-verbal gestures of the Greta virtual agent and of the Nao physical robot.

[1]  Charles Rich,et al.  Using the Behavior Markup Language for human-robot interaction , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  K. Scherer,et al.  Bodily expression of emotion , 2009 .

[3]  Pierre Blazevic,et al.  Mechatronic design of NAO humanoid , 2009, 2009 IEEE International Conference on Robotics and Automation.

[4]  Maurizio Mancini,et al.  The FML-APML language , 2008 .

[5]  Stefan Kopp,et al.  MODELING THE PRODUCTION OF COVERBAL ICONIC GESTURES BY LEARNING BAYESIAN DECISION NETWORKS , 2010, Appl. Artif. Intell..

[6]  Mark Steedman,et al.  APML, a Markup Language for Believable Behavior Generation , 2004, Life-like characters.

[7]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[8]  Maurizio Mancini,et al.  Implementing Expressive Gesture Synthesis for Embodied Conversational Agents , 2005, Gesture Workshop.

[9]  Elisabeth André,et al.  Creation and Evaluation of emotion expression with body movement, sound and eye color for humanoid robots , 2011, 2011 RO-MAN.

[10]  Stefan Kopp,et al.  The Next Step towards a Function Markup Language , 2008, IVA.

[11]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[12]  Matthew Stone,et al.  Speaking with hands: creating animated conversational characters from recordings of human performance , 2004, ACM Trans. Graph..

[13]  Gregory M. P. O'Hare,et al.  Where Robots and Virtual Agents Meet , 2009, Int. J. Soc. Robotics.

[14]  Norman I. Badler,et al.  The EMOTE model for effort and shape , 2000, SIGGRAPH.

[15]  Pengcheng Luo,et al.  Synchronized gesture and speech production for humanoid robots , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Cynthia Breazeal,et al.  Emotion and sociable humanoid robots , 2003, Int. J. Hum. Comput. Stud..

[17]  Catherine Pelachaud,et al.  Multimodal expressive embodied conversational agents , 2005, ACM Multimedia.

[18]  Stefan Kopp,et al.  Generation and Evaluation of Communicative Robot Gesture , 2012, Int. J. Soc. Robotics.

[19]  Catherine Pelachaud,et al.  Design and implementation of an expressive gesture model for a humanoid robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[20]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[21]  Catherine Pelachaud,et al.  Gestural mind markers in ECAs , 2003, AAMAS '03.

[22]  Hans-Peter Seidel,et al.  Annotated New Text Engine Animation Animation Lexicon Animation Gesture Profiles MR : . . . JL : . . . Gesture Generation Video Annotated Gesture Script , 2007 .

[23]  Jacob W. Crandall,et al.  Expressing emotions through robots: A case study using off-the-shelf programming interfaces , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[24]  H. Iba,et al.  Humanoid robot presentation controlled by multimodal presentation markup language MPML , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[25]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[26]  De Ruiter,et al.  Gesture and speech production , 1998 .

[27]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.