Modeling Gaze Behavior as a Function of Discourse Structure

This paper addresses the problem of designing conversational agents that exhibit appropriate gaze behavior during dialogues with human users. Previous research on gaze behavior has concentrated on its relationship to turn-taking phenomena [3,4,5]. Recent work has incorporated some of these findings into the design of autonomous human-like conversational agents and interactive communicative humanoids [1,13]. However, none of this research has examined the relationship between information structure and gaze behavior. In this paper we discuss why turn-taking is not an adequate explanation for gaze behavior in conversation and why information structure should be integrated with turn-taking as an explanation for this behavior. We then examine the relationship of gaze behavior to information structure and turn-taking through an empirical analysis of discourse transcripts for several dyadic conversations. A simple algorithm for assigning gaze behavior is proposed on the basis of the findings of this empirical analysis. We describe work in progress on implementing this algorithm in an existent communicative humanoid agent [14] with the goal of producing more natural gaze behavior related to propositional content in human-computer conversations.

[1]  D. Crystal,et al.  Intonation and Grammar in British English , 1967 .

[2]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[3]  S. Duncan,et al.  On the structure of speaker–auditor interaction during speaking turns , 1974, Language in Society.

[4]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[5]  S. Whittaker,et al.  Cues and control in Expert-Client Dialogues , 1988, Annual Meeting of the Association for Computational Linguistics.

[6]  David Frohlich,et al.  Computers and conversation , 1990 .

[7]  Enric Vallduví,et al.  The Informational Component , 1990 .

[8]  Akikazu Takeuchi,et al.  Communicative facial displays as a new conversational modality , 1993, INTERCHI.

[9]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[10]  Harvey Sacks,et al.  Lectures on Conversation , 1995 .

[11]  J. Cassell,et al.  Communicative humanoids: a computational model of psychosocial dialogue skills , 1996 .

[12]  Scott Prevost.  An Information Structural Approach to Spoken Language Generation , 1996, ACL.

[13]  Mark Steedman,et al.  Generating Facial Expressions for Speech , 1996, Cogn. Sci..

[14]  Kristinn R. Thórisson,et al.  Why Put An Agent In A Human Body: The Importance Of Communicative Feedback in Human-Humanoid Dialogu , 1996 .

[15]  Kristinn R. Thórisson.  Gandalf: an embodied humanoid capable of real-time multimodal dialogue with people , 1997, AGENTS '97.