Towards a simulation of conversations with expressive embodied speakers and listeners

In this paper we present some results to model complex interactions among virtual characters that participate in negotiation dialogues as well as our work related to a gaze model that controls the eye behavior of several agents conversing with each other. As a test-bed we have created an Avatar Arena in which several avatars negotiate on meeting arrangement tasks on behalf of their users. To enhance the naturalness of the emerging negotiation dialogues we need to determine both the behaviors of speakers as well as the behaviors of listening characters. In our approach we try to exploit socio-physiological concepts, such as cognitive balance and dissonance to determine verbal and nonverbal behavior of all dialogue participants. We pay particular attention to the gaze behavior of the speaker considering the communicative functions the speaker desires to communicate as well as a statistical model of eye movements. In addition, a model of gaze behavior for listeners is also proposed.

[1]  Norman I. Badler,et al.  Eyes alive , 2002, ACM Trans. Graph..

[2]  Randall W. Hill,et al.  Toward the holodeck: integrating graphics, sound, character and story , 2001, AGENTS '01.

[3]  Kristinn R. Thórisson,et al.  Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action , 2002 .

[4]  Catherine Pelachaud,et al.  Rules for Responsive Robots: Using Human Interactions to Build Virtual Interactions , 2002 .

[5]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[6]  J. Cassell,et al.  Turn taking vs. Discourse Structure: How Best to Model Multimodal Conversation , 1998 .

[7]  Thomas Rist,et al.  Controlling the Behavior of Animated Presentation Agents in the Interface: Scripting versus Instructing , 2001, AI Mag..

[8]  I. Poggi.  MIND MARKERS , 2003 .

[9]  Catherine Pelachaud,et al.  Embodied contextual agent in information delivering application , 2002, AAMAS '02.

[10]  L. Festinger,et al.  A Theory of Cognitive Dissonance , 2017 .

[11]  C. Osgood,et al.  The principle of congruity in the prediction of attitude change. , 1955, Psychological review.

[12]  J. Breese,et al.  Emotion and personality in a conversational agent , 2001 .

[13]  Catherine Pelachaud,et al.  Eye Communication in a Conversational 3D Synthetic Agent , 2000, AI Commun..

[14]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[15]  Jonas Beskow,et al.  Developing a 3D-agent for the august dialogue system , 1999, AVSP.

[16]  David R. Traum,et al.  Embodied agents for multi-party dialogue in immersive virtual worlds , 2002, AAMAS '02.

[17]  F. Heider.  The psychology of interpersonal relations , 1958 .

[18]  Norihiro Hagita,et al.  Messages embedded in gaze of interface agents --- impression management with agent's gaze , 2002, CHI.

[19]  James C. Lester,et al.  Deictic and emotive communication in animated pedagogical agents , 2001 .

[20]  Thomas Rist,et al.  Presenting through performing: on the use of multiple lifelike characters in knowledge-based presentation systems , 2000, IUI '00.

[21]  Mitsuru Ishizuka,et al.  Social role awareness in animated agents , 2001, AGENTS '01.

[22]  Jonas Beskow,et al.  Animation of talking agents , 1997, AVSP.

[23]  James M. Rehg,et al.  Computer Vision for Human–Machine Interaction: Visual Sensing of Humans for Active Public Interfaces , 1998 .

[24]  K.R. Thorisson,et al.  Layered modular action control for communicative humanoids , 1997, Proceedings. Computer Animation '97 (Cat. No.97TB100120).

[25]  S. Drucker,et al.  The Role of Eye Gaze in Avatar Mediated Conversational Interfaces , 2000 .

[26]  Norman I. Badler,et al.  Where to Look? Automating Attending Behaviors of Virtual Human Characters , 1999, Agents.