Gaze transfer in remote cooperation: Is it always helpful to see what your partner is attending to?

Establishing common ground in remote cooperation is challenging because nonverbal means of ambiguity resolution are limited. In such settings, information about a partner's gaze can support cooperative performance, but it is not yet clear whether and to what extent the abundance of information reflected in gaze comes at a cost. Specifically, in tasks that mainly rely on spatial referencing, gaze transfer might be distracting and leave the partner uncertain about the meaning of the gaze cursor. To examine this question, we let pairs of participants perform a joint puzzle task. One partner knew the solution and instructed the other partner's actions by (1) gaze, (2) speech, (3) gaze and speech, or (4) mouse and speech. Based on these instructions, the acting partner moved the pieces under conditions of high or low autonomy. Performance was better when using either gaze or mouse transfer compared to speech alone. However, in contrast to the mouse, gaze transfer induced uncertainty, evidenced in delayed responses to the cursor. Also, participants tried to resolve ambiguities by engaging in more verbal effort, formulating more explicit object descriptions and fewer deictic references. Thus, gaze transfer seems to increase uncertainty and ambiguity, thereby complicating grounding in this spatial referencing task. The results highlight the importance of closely examining task characteristics when considering gaze transfer as a means of support.

[1]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[2]  A. Almor,et al.  Noun-phrase anaphors and focus: the informational load hypothesis. , 1999, Psychological review.

[3]  David G. Novick,et al.  Conversational effectiveness in multimedia communications , 1995, Inf. Technol. People.

[4]  Zenzi M. Griffin,et al.  PSYCHOLOGICAL SCIENCE Research Article WHAT THE EYES SAY ABOUT SPEAKING , 2022 .

[5]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[6]  J. Helmert,et al.  Following closely ? The effects of viewing conditions on gaze versus mouse transfer in remote cooperation , 2011 .

[7]  Roel Vertegaal,et al.  Catching the eye: management of joint attention in cooperative work , 1997, SGCH.

[8]  W. Chafe.  Givenness, contrastiveness, definiteness, subjects, topics, and point of view , 1976 .

[9]  Martin J Pickering,et al.  The use of visual context during the production of referring expressions , 2010, Quarterly journal of experimental psychology.

[10]  Susan Brennan,et al.  Another person's eye gaze as a cue in solving programming problems , 2004, ICMI '04.

[11]  Daniel C. Richardson,et al.  The Art of Conversation Is Coordination , 2007, Psychological science.

[12]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[13]  Christopher A. Dickinson,et al.  Coordinating spatial referencing using shared gaze , 2010, Psychonomic bulletin & review.

[14]  David J. Manning,et al.  Learning from others: effects of viewing another person's eye movements while searching for chest nodules , 2008, SPIE Medical Imaging.

[15]  Zenzi M. Griffin,et al.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[16]  Jan Theeuwes,et al.  Capture of the eyes by relevant and irrelevant onsets , 2007, Experimental Brain Research.

[17]  B. Velichkovsky,et al.  Eye typing in application: A comparison of two interfacing systems with ALS patients , 2008 .

[18]  John Paulin Hansen,et al.  Eye-gaze control of multimedia systems , 1995 .

[19]  Boris M. Velichkovsky,et al.  Influences of dwell time and cursor control on the performance in gaze driven typing , 2008 .

[20]  A. Monk,et al.  A Look Is Worth a Thousand Words: Full Gaze Awareness in Video-Mediated Conversation , 2002 .

[21]  John Paulin Hansen,et al.  Gaze-guided viewing of interactive movies , 2005, Digit. Creativity.

[22]  Anand K. Gramopadhye,et al.  Evaluating Different Display Techniques for Communicating Search Strategy Training in a Collaborative Virtual Aircraft Inspection Environment , 2005 .

[23]  Shumin Zhai,et al.  RealTourist - A Study of Augmenting Human-Human and Human-Computer Dialogue with Eye-Gaze Overlay , 2005, INTERACT.

[24]  WhittakerSteve.  Rethinking video as a technology for interpersonal communications , 1995 .

[25]  Victor S. Ferreira,et al.  How do speakers avoid ambiguous linguistic expressions? , 2005, Cognition.

[26]  Jean Carletta,et al.  Eyetracking for two-person tasks with manipulation of a virtual world , 2010, Behavior research methods.

[27]  B. Velichkovsky.  Communicating attention: Gaze position transfer in cooperative problem solving , 1995 .

[28]  Richard A. Bolt,et al.  A gaze-responsive self-disclosing display , 1990, CHI '90.

[29]  Z. Griffin.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[30]  Christopher A. Dickinson,et al.  Coordinating cognition: The costs and benefits of shared gaze during collaborative search , 2008, Cognition.

[31]  Boris M. Velichkovsky,et al.  Towards gaze-mediated interaction: Collecting solutions of the "Midas touch problem" , 1997, INTERACT.

[32]  S. Brennan,et al.  Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation , 2007 .

[33]  Päivi Majaranta,et al.  Twenty years of eye typing: systems and design issues , 2002, ETRA.

[34]  B. Velichkovsky.  Heterarchy of cognition: The depths and the highs of a framework for memory research , 2002, Memory.

[35]  Manpreet Kaur,et al.  Where is "it"? Event Synchronization in Gaze-Speech Input Systems , 2003, ICMI '03.

[36]  Kristinn R. Thórisson,et al.  Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[37]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[38]  Steve Whittaker,et al.  Rethinking video as a technology for interpersonal communications: theory and design implications , 1995, Int. J. Hum. Comput. Stud..

[39]  Max-Planck-Gesellschaft zur Förderung der Wissenschaften,et al.  The Child's construction of language , 1981 .

[40]  Gwyneth Doherty-Sneddon,et al.  Face-to-face and video mediated communication: a comparison of dialogue structure and task performance , 1997 .