Designing Socially Aware Conversational Agents

In this paper we address the problem of how to make conversational agents socially aware. State-of-the-art conversational agents cannot deal with multi-user situations where the user-system dialog is interleaved with discussions between users. We describe the development of an algorithm for determining addressee-hood of user utterances. The algorithm makes errors in determining addressee-hood for individual utterances, classifying utterances intended for the system as utterances intended for the other user and the other way around, creating unexpected situations in the agent's behaviour. This raises the question of how to design the conversational agent so that the users understand why the agent behaves in particular ways. We describe a study aimed at obtaining guidelines for the design of a socially aware conversational agent. We conclude that principles for modelling the behaviour of the agent are to be derived from theories about human communication rather than from theories about human-computer interaction.

[1]  Tanja Schultz,et al.  Identifying the addressee in human-human-robot interactions based on head pose and speech , 2004, ICMI '04.

[2]  Naoki Mukawa,et al.  Video cut editing rule based on participants' gaze in multiparty conversation , 2003, MULTIMEDIA '03.

[3]  J. Cassell,et al.  Turn Taking versus Discourse Structure , 1999 .

[4]  Janienke Sturm,et al.  Effects of prolonged use on the usability of a multimodal form-filling interface , 2004 .

[5]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[6]  Jacques M. B. Terken,et al.  Facial Orientation During Multi-party Interaction with Information Kiosks , 2003, INTERACT.

[7]  Jacques M. B. Terken,et al.  Multimodalcues for addressee-hood in triadic communication with a human information retrieval agent , 2007, ICMI '07.

[8]  Janienke Sturm,et al.  Usability Evaluation of a Dutch Multimodal System for Train Timetable Information , 2002, LREC.

[9]  Berry Eggen,et al.  Identifying the intended addressee in mixed human-human and human-computer interaction from non-verbal features , 2005, ICMI '05.

[10]  van Kg Koen Turnhout.  Socially aware conversational agents , 2007 .

[11]  Michael Argyle,et al.  The central Europe experiment: Looking at persons and looking at objects , 1976 .

[12]  Yorick Wilks.  Machine Conversations , 1999 .

[13]  Ben Shneiderman,et al.  Designing the user interface (2nd ed.): strategies for effective human-computer interaction , 1992 .

[14]  Anton Nijholt,et al.  Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes , 2001, CHI.

[15]  Laila Dybkjær,et al.  Spoken Multimodal Human-Computer Dialogue in Mobile Environments , 2005 .

[16]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[17]  Kristinn R. Thórisson,et al.  Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action , 2002 .

[18]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[19]  Björn Granström,et al.  Multimodality in Language and Speech Systems , 2002 .