Conversational gaze mechanisms for humanlike robots

During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.

[1]  K. Williams,et al.  Cyberostracism: effects of being ignored over the Internet. , 2000, Journal of personality and social psychology.

[2]  Justine Cassell,et al.  BodyChat: autonomous communicative behaviors in avatars , 1998, AGENTS '98.

[3]  Anthony Steed,et al.  An assessment of eye-gaze potential within immersive virtual environments , 2007, TOMCCAP.

[4]  M. Turk,et al.  Transformed social interaction, augmented gaze, and social influence in immersive virtual environments , 2005 .

[5]  Roel Vertegaal,et al.  Effects of Gaze on Multiparty Mediated Communication , 2000, Graphics Interface.

[6]  R. Kleck,et al.  Congruence between the indicative and communicative functions of eye contact in interpersonal relations. , 1968, The British journal of social and clinical psychology.

[7]  Kristinn R. Thórisson,et al.  Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action , 2002 .

[8]  E. Goffman.  On face-work; an analysis of ritual elements in social interaction. , 1955, Psychiatry.

[9]  S. Brennan,et al.  Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation , 2007 .

[10]  W. L. Libby,et al.  Eye contact and direction of looking as stable individual differences. , 1970 .

[11]  J. Cassell,et al.  Turn Taking versus Discourse Structure , 1999 .

[12]  S. Maynard.  On back-channel behavior in Japanese and English casual conversation , 1987 .

[13]  John A. Johnson,et al.  The international personality item pool and the future of public-domain personality measures ☆ , 2006 .

[14]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[15]  Hideaki Kuzuoka,et al.  Museum guide robot based on sociological interaction analysis , 2007, CHI.

[16]  A. Aron,et al.  Inclusion of Other in the Self Scale and the structure of interpersonal closeness , 1992 .

[17]  Brenda Laurel,et al.  Computers as theatre , 1991 .

[18]  C. Goodwin.  Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn‐Beginning , 1980 .

[19]  Steve Whittaker,et al.  Cues and control in Expert-Client Dialogues , 1988, ACL.

[20]  H. H. Clark,et al.  Hearers and speech acts , 1982 .

[21]  J. P. Otteson,et al.  Effect of Teacher's Gaze on Children's Story Recall , 1980 .

[22]  Charles A. Bouman,et al.  CLUSTER: An Unsupervised Algorithm for Modeling Gaussian Mixtures , 2014 .

[23]  J. S. Efran.  Looking for approval: effects on visual behavior of approbation from persons differing in importance. , 1968, Journal of personality and social psychology.

[24]  Dirk Heylen,et al.  CONTROLLING THE GAZE OF CONVERSATIONAL AGENTS , 2005 .

[25]  R. Exline.  Explorations in the process of person perception: visual interaction in relation to competition, sex, and need for affiliation , 1963 .

[26]  H. H. Clark.  Arenas of language use , 1993 .

[27]  E. Schegloff,et al.  Opening up Closings , 1973 .

[28]  Francis K. H. Quek,et al.  Gesture, speech, and gaze cues for discourse segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[29]  E. Schegloff.  Overlapping talk and the organization of turn-taking for conversation , 2000, Language in Society.

[30]  R. Bales,et al.  Personality and Interpersonal Behavior. , 1971 .

[31]  C. Goodwin.  Conversational Organization: Interaction Between Speakers and Hearers , 1981 .

[32]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[33]  Hideaki Kuzuoka,et al.  Precision timing in human-robot interaction: coordination of head movement and utterance , 2008, CHI.

[34]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[35]  Carole Edelsky.  Who's got the floor? , 1981, Language in Society.

[36]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[37]  V. Yngve.  On getting a word in edgewise , 1970 .

[38]  Herbert H. Clark,et al.  Coordinating beliefs in conversation , 1992 .

[39]  Bilge Mutlu,et al.  A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[40]  Norman I. Badler,et al.  Eyes alive , 2002, ACM Trans. Graph..

[41]  Stephen C. Levinson,et al.  Putting linguistics on a proper footing: Explorations in Goffman's participation framework , 1988 .

[42]  Mel Slater,et al.  The impact of eye gaze on communication using humanoid avatars , 2001, CHI.

[43]  Tetsuo Ono,et al.  Robovie: an interactive humanoid robot , 2001 .

[44]  S. Drucker,et al.  The Role of Eye Gaze in Avatar Mediated Conversational Interfaces , 2000 .

[45]  Anton Nijholt,et al.  Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes , 2001, CHI.

[46]  Julia Hirschberg,et al.  Intonational Features of Local and Global Discourse Structure , 1992, HLT.

[47]  Elisabeth André,et al.  Where Do They Look? Gaze Behaviors of Multiple Users Interacting with an Embodied Conversational Agent , 2005, IVA.

[48]  D. Crystal,et al.  Intonation and Grammar in British English , 1967 .

[49]  William D. Smart,et al.  What Can Actors Teach Robots About Interaction? , 2010, AAAI Spring Symposium: It's All in the Timing.

[50]  F. Thomas,et al.  The illusion of life : Disney animation , 1981 .

[51]  D. Perez-Granados.  CHI Workshop Shaping Human-Robot Interaction Understanding the Social Aspects of Intelligent , 2022 .

[52]  Lee Sproull,et al.  My partner is a real dog: cooperation with social agents , 1996, CSCW '96.

[53]  Penelope Brown,et al.  Politeness: Some Universals in Language Usage , 1989 .

[54]  M. Argyle,et al.  Gaze, Mutual Gaze, and Proximity , 1972 .

[55]  Kyoko Inoue,et al.  Aspects of Japanese Discourse Structure , 1979 .

[56]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.

[57]  Robin Wolff,et al.  Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments , 2008, CSCW.

[58]  Ning Wang,et al.  Experimental evaluation of polite interaction tactics for pedagogical agents , 2005, IUI.

[59]  E. Goffman.  Relations in Public: Microstudies of the Public Order , 1971 .

[60]  D. Hymes.  Models of the Interaction of Language and Social Life , 2009 .

[61]  Stephen C. Levinson,et al.  Putting linguistics on a proper footing: Explorations in Goffman's concepts of participation. , 1988 .

[62]  E. Schegloff.  Sequencing in Conversational Openings , 1968 .

[63]  K. Yamazaki,et al.  Coordination of verbal and non-verbal actions in human―robot interaction at museums and exhibitions , 2010 .

[64]  R. Bales,et al.  Channels of communication in small groups. , 1951 .

[65]  Julia Hirschberg,et al.  The intonational Structuring of Discourse , 1986, ACL.

[66]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[67]  Yukiko I. Nakano,et al.  Non-Verbal Cues for Discourse Structure , 2022 .

[68]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[69]  J. Cassell,et al.  Turn taking vs. Discourse Structure: How Best to Model Multimodal Conversation , 1998 .

[70]  Gamini Dissanayake,et al.  Nonverbal robot-group interaction using an imitated gaze cue , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[71]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[72]  Ning Wang,et al.  The Politeness Effect in an Intelligent Foreign Language Tutoring System , 2008, Intelligent Tutoring Systems.

[73]  R. Hayashi,et al.  Simultaneous talk—from the perspective of floor management of English and Japanese speakers , 1988 .

[74]  Raj M. Ratwani,et al.  Integrating vision and audition within a cognitive architecture to track conversations , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[75]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.

[76]  Gillian Brown,et al.  Questions of intonation , 1980 .

[77]  M. Crocker,et al.  Investigating joint attention mechanisms through spoken human–robot interaction , 2011, Cognition.

[78]  J. V. Sherwood.  Facilitative Effects of Gaze upon Learning , 1987 .

[79]  Sven Behnke,et al.  Towards a humanoid museum guide robot that interacts with multiple persons , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[80]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.

[81]  Margaret L. McLaughlin,et al.  AWKWARD SILENCES: BEHAVIORAL ANTECEDENTS AND CONSEQUENCES OF THE CONVERSATIONAL LAPSE , 1982 .

[82]  Mariette DiChristina.  Look of Love , 2010 .

[83]  O. Watson,et al.  Proxemic Behavior: A Cross-Cultural Study. , 1971 .

[84]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[85]  W. Hanks.  Language & communicative practices , 1995 .

[86]  Jacqueline M. C. Smith,et al.  The role of gaze in impression formation. , 1975, The British journal of social and clinical psychology.

[87]  KandaTakayuki,et al.  Conversational gaze mechanisms for humanlike robots , 2012 .

[88]  Andrei Popescu-Belis,et al.  What are discourse markers ? , 2003 .

[89]  D. Tannen.  Conversational Style: Analyzing Talk Among Friends , 1984 .

[90]  Hiroko Tanaka,et al.  Turn-taking in Japanese conversation : a study in grammar and interaction , 2000 .

[91]  Senko Kumiya Maynard,et al.  Japanese Conversation: Self-Contextualization Through Structure and Interactional Management , 1989 .

[92]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[93]  Nigel G. Ward,et al.  Prosodic features which cue back-channel responses in English and Japanese , 2000 .

[94]  James H. Wirth,et al.  Eye Gaze as Relational Evaluation: Averted Eye Gaze Leads to Feelings of Ostracism and Relational Devaluation , 2010, Personality & social psychology bulletin.