Deixis and gaze in collaborative work at a distance (over a shared map): a computational model to detect misunderstandings

This paper presents an algorithm that detects misunderstandings in collaborative work at a distance. It analyses the movements of collaborators' eyes on the shared workspace, their utterances containing references about this workspace, and the availability of 'remote' deictic gestures. This method is based on two findings: 1. participants look at the points they are talking about in their message; 2. their gazes are more dense around these points compared to other random looks in the same timeframe. The algorithm associates the distance between the gazes of the emitter and gazes of the receiver of a message with the probability that the recipient did not understand the message.

[1]  H. H. Clark,et al.  Speaking while monitoring addressees for understanding , 2004 .

[2]  BONNIE A. NARDI,et al.  Beyond Bandwidth: Dimensions of Connection in Interpersonal Communication , 2005, Computer Supported Cooperative Work (CSCW).

[3]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[4]  Carl Gutwin,et al.  The effects of workspace awareness support on the usability of real-time distributed groupware , 1999, TCHI.

[5]  A. Monk,et al.  A Look Is Worth a Thousand Words: Full Gaze Awareness in Video-Mediated Conversation , 2002 .

[6]  Susan R. Fussell,et al.  Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks , 2004, Hum. Comput. Interact..

[7]  Hiroshi Ishii,et al.  ClearBoard: a seamless medium for shared drawing and conversation with eye contact , 1992, CHI.

[8]  Susan R. Fussell,et al.  Analyzing and predicting focus of attention in remote collaborative tasks , 2005, ICMI '05.

[9]  S. Greenberg,et al.  The Importance of Awareness for Team Cognition in Distributed Collaboration , 2001 .

[10]  B. Velichkovsky.  Communicating attention: Gaze position transfer in cooperative problem solving , 1995 .

[11]  Shumin Zhai,et al.  RealTourist - A Study of Augmenting Human-Human and Human-Computer Dialogue with Eye-Gaze Overlay , 2005, INTERACT.

[12]  Tom Rodden,et al.  Turn it this way: grounding collaborative action with remote gestures , 2007, CHI.

[13]  Zenzi M. Griffin,et al.  PSYCHOLOGICAL SCIENCE Research Article WHAT THE EYES SAY ABOUT SPEAKING , 2022 .

[14]  Roel Vertegaal,et al.  The GAZE groupware system: mediating joint attention in multiparty communication and collaboration , 1999, CHI '99.

[15]  Claude Vandeloise,et al.  Spatial Prepositions: A Case Study from French , 1991 .

[16]  Douglas DeCarlo,et al.  Robust clustering of eye movement recordings for quantification of visual interest , 2004, ETRA.

[17]  Pierre Dillenbourg,et al.  The effects of explicit referencing in distance problem solving over shared maps , 2007, GROUP.

[18]  Steve Whittaker,et al.  Shared Workspaces: How Do They Work and When Are They Useful? , 1993, Int. J. Man Mach. Stud..

[19]  Gerd Kortuem,et al.  "Where are you pointing at?" A study of remote collaboration in a wearable videoconference system , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[20]  Dario D. Salvucci.  Inferring intent in eye-based interfaces: tracing eye movements with process models , 1999, CHI '99.

[21]  喜多 壮太郎.  Pointing : where language, culture, and cognition meet , 2013 .

[22]  Martin Wessner,et al.  Explicit referencing in chat supports collaborative learning , 2005, CSCL.

[23]  Stephanie D. Teasley,et al.  Perspectives on socially shared cognition , 1991 .

[24]  S. Brennan.  Seeking and providing evidence for mutual understanding , 1990 .

[25]  Robert E. Kraut,et al.  Visual Information as a Conversational Resource in Collaborative Physical Tasks , 2003, Hum. Comput. Interact..

[26]  Daniel C. Richardson,et al.  Looking To Understand: The Coupling Between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension , 2005, Cogn. Sci..

[27]  Robert E. Kraut,et al.  The use of visual information in shared visual spaces: informing the development of virtual co-presence , 2002, CSCW '02.