"I simply watched where she was looking at"

Mixed reality (MR) cooperation scenarios are more and more interesting for business and research as powerful wearable devices like head mounted displays (HMD) become commercially available. A lot of work focuses on remote MR cooperation settings like remote maintenance support and co-located scenarios in which participants cooperate over a longer period time. Despite this, MR also has great potential for real-time co-located cooperation support with the need of short-term decisions and interactions. However, little is known on how this support can be provided. To bridge this gap, we conducted an experiment using a MR visual search task performed by dyads. Based on related work, visual search was chosen to represent typical challenges of short-term cooperative MR tasks. The aim of the experiment was to explore how the participants coordinate their searches and how this influences their performance in a task. We found that participants mainly used embodied and verbal cues to coordinate their searches (rather than virtual cues provided by the HMD) and that less communication worked significantly better, which is in (partial) contrast to existing findings. We discuss potential reasons for and impacts of these findings.

[1]  Harri Siirtola,et al.  Shared Gaze in Collaborative Visual Search , 2019, Int. J. Hum. Comput. Interact..

[2]  Nicholas A. Giovinco,et al.  A Heads-Up Display for Diabetic Limb Salvage Surgery , 2014, Journal of diabetes science and technology.

[3]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[4]  Roman Rädle,et al.  Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load , 2016, CHI.

[5]  Scott Bateman,et al.  Stabilized Annotations for Mobile Remote Assistance , 2016, CHI.

[6]  Boris M. Velichkovsky,et al.  Gaze transfer in remote cooperation: Is it always helpful to see what your partner is attending to? , 2013, Quarterly journal of experimental psychology.

[7]  Susan Brennan,et al.  Another person's eye gaze as a cue in solving programming problems , 2004, ICMI '04.

[8]  Robert W. Lindeman,et al.  Augmented reality annotation for social video sharing , 2016, SIGGRAPH ASIA Mobile Graphics and Interactive Applications.

[9]  Hans-Werner Gellersen,et al.  Visual Attention-Based Access , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[10]  Kjeld Schmidt,et al.  Taking CSCW seriously , 1992, Computer Supported Cooperative Work (CSCW).

[11]  Thomas Hermann,et al.  Collaboration in Augmented Reality: How to Establish Coordination and Joint Attention? , 2011, ECSCW.

[12]  Dieter Schmalstieg,et al.  The Studierstube Augmented Reality Project , 2002, Presence: Teleoperators & Virtual Environments.

[13]  Yanxia Zhang,et al.  Look together: using gaze for assisting co-located collaborative search , 2017, Personal and Ubiquitous Computing.

[14]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[15]  Kristine Nagel,et al.  Understanding the design space of referencing in collaborative augmented reality environments , 2007, GI '07.

[16]  Anoop Gupta,et al.  Communication behaviors of co-located users in collaborative AR interfaces , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[17]  C. Kleinke Gaze and eye contact: a research review. , 1986, Psychological bulletin.

[18]  T. Ronald,et al.  Azuma A Survey of Augmented Reality , 2022 .

[19]  Dragos Datcu,et al.  Handheld Augmented Reality for Distributed Collaborative Crime Scene Investigation , 2016, GROUP.

[20]  Carl Gutwin,et al.  A Descriptive Framework of Workspace Awareness for Real-Time Groupware , 2002, Computer Supported Cooperative Work (CSCW).

[21]  Gerd Kortuem,et al.  "Where are you pointing at?" A study of remote collaboration in a wearable videoconference system , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[22]  Robert J. Moore,et al.  Doing Virtually Nothing: Awareness and Accountability in Massively Multiplayer Online Worlds , 2007, Computer Supported Cooperative Work (CSCW).

[23]  Miguel P. Eckstein,et al.  Attention Allocation Aid for Visual Search , 2017, CHI.

[24]  Lucy Suchman,et al.  Supporting Articulation Work , 1995, Computerization and Controversy, 2nd Ed..

[25]  H. Müller,et al.  Visual search and selective attention , 2006 .

[26]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[27]  Christopher A. Dickinson,et al.  Coordinating cognition: The costs and benefits of shared gaze during collaborative search , 2008, Cognition.

[28]  Ina Bornkessel-Schlesewsky,et al.  A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks , 2018, IEEE Transactions on Visualization and Computer Graphics.

[29]  Darren Gergle,et al.  Designing Shared Gaze Awareness for Remote Collaboration , 2016, CSCW Companion.

[30]  Steve Benford,et al.  Fragmented interaction: establishing mutual orientation in virtual environments , 1998, CSCW '98.

[31]  Stephan Lukosch,et al.  As if being there: mediated reality for crime scene investigation , 2012, CSCW '12.

[32]  Bilge Mutlu,et al.  Handheld or Handsfree?: Remote Collaboration via Lightweight Head-Mounted Displays and Handheld Devices , 2015, CSCW.

[33]  Kjeld Schmidt,et al.  Taking CSCW Seriously: Supporting Articulation Work * , 1992 .

[34]  Tobias Höllerer,et al.  The Effects of Visual Realism on Search Tasks in Mixed Reality Simulation , 2013, IEEE Transactions on Visualization and Computer Graphics.

[35]  Arindam Dey,et al.  The Effects of Sharing Awareness Cues in Collaborative Mixed Reality , 2019, Front. Robot. AI.

[36]  Ke Huo,et al.  SynchronizAR: Instant Synchronization for Spontaneous and Spatial Collaborations in Augmented Reality , 2018, UIST.

[37]  Ratnesh Chaturvedi,et al.  A Review on the use of Augmented Reality to Generate Safety Awareness and Enhance Emergency Response , 2016 .

[38]  Darren Gergle,et al.  Visualizing Gaze Information from Multiple Students to Support Remote Instruction , 2018, CHI Extended Abstracts.

[39]  Egui Zhu,et al.  Augmented reality in healthcare education: an integrative review , 2014, PeerJ.

[40]  Michael Lankes,et al.  Animating Participants in Co-located Playful Mixed-Reality Installations , 2018, 2018 IEEE 1st Workshop on Animation in Virtual and Augmented Environments (ANIVAE).

[41]  Carl Gutwin,et al.  The Emergence of High-Speed Interaction and Coordination in a (Formerly) Turn-based Groupware Game , 2016, GROUP.

[42]  Hirokazu Kato,et al.  Transferability of Spatial Maps: Augmented Versus Virtual Reality Training , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[43]  Kristine Lund,et al.  The importance of gaze and gesture in interactive multimodal explanation , 2007, Lang. Resour. Evaluation.

[44]  Patrick Baudisch,et al.  Halo: a technique for visualizing off-screen objects , 2003, CHI '03.

[45]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[46]  Mark Billinghurst,et al.  Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration , 2016, IEEE Transactions on Visualization and Computer Graphics.

[47]  Tobias Höllerer,et al.  Augmented reality: principles and practice , 2016, SIGGRAPH Courses.

[48]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[49]  Cristina V. Lopes,et al.  A Spatial Augmented Reality Rehab System for Post-Stroke Hand Rehabilitation , 2013, MMVR.

[50]  Tobias Höllerer,et al.  World-stabilized annotations and virtual scene navigation for remote collaboration , 2014, UIST.

[51]  Roel Vertegaal,et al.  The GAZE groupware system: mediating joint attention in multiparty communication and collaboration , 1999, CHI '99.

[52]  M. Pomplun,et al.  The effects of task difficulty on visual search strategy in virtual 3D displays. , 2013, Journal of vision.

[53]  Alain Karsenty,et al.  Transparency and awareness in a real-time groupware system , 1992, UIST '92.

[54]  Mark Billinghurst,et al.  Pinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality , 2018, CHI.