Effect of Visual Cues on Pointing Tasks in Co-located Augmented Reality Collaboration

Visual cues are essential in computer-mediated communication. It is especially important when communication happens in a collaboration scenario that requires focusing several users’ attention on a specific object among other similar ones. This paper explores the effect of visual cues on pointing tasks in co-located Augmented Reality (AR) collaboration. A user study (N = 32, 16 pairs) was conducted to compare two types of visual cues: Pointing Line (PL) and Moving Track (MT). Both are head-based visual techniques. Through a series of collaborative pointing tasks on objects with different states (static and dynamic) and density levels (low, medium and high), the results showed that PL was better on task performance and usability, but MT was rated higher on social presence and user preference. Based on our results, some design implications are provided for pointing tasks in co-located AR collaboration.

[1]  Gerard Jounghyun Kim,et al.  Effects of avatar and background representation forms to co-presence in mixed reality (MR) tele-conference systems , 2016, SIGGRAPH ASIA Virtual Reality meets Physical Reality.

[2]  Difeng Yu,et al.  iText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality Systems , 2021, UIST.

[3]  John C. Tang,et al.  Videodraw: a video interface for collaborative drawing , 1991, TOIS.

[4]  Stephan Lukosch,et al.  Collaboration in Augmented Reality , 2015, Computer Supported Cooperative Work (CSCW).

[5]  Katashi Nagao,et al.  The world through the computer: computer augmented interaction with real world environments , 1995, UIST '95.

[6]  Carl Gutwin,et al.  Workspace Awareness in Real-Time Distributed Groupware: Framework, Widgets, and Evaluation , 1996, BCS HCI.

[7]  Miguel Antunes,et al.  An abstraction for awareness management in collaborative virtual environments , 2001, VRST '01.

[8]  Susan R. Fussell,et al.  Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks , 2004, Hum. Comput. Interact..

[9]  Seungwon Kim,et al.  Comparing pointing and drawing for remote collaboration , 2013, ISMAR.

[10]  Carl Gutwin,et al.  Semantic telepointers for groupware , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.

[11]  Matt Adcock,et al.  Hand gestures and visual annotation in live 360 panorama-based mixed reality remote collaboration , 2018, OZCHI.

[12]  Joseph J. LaViola,et al.  Collaborative Conceptual Modeling Using the SKETCH Framework , 1998 .

[13]  Kristine Nagel,et al.  Understanding the design space of referencing in collaborative augmented reality environments , 2007, GI '07.

[14]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[15]  Mark Billinghurst,et al.  Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration , 2016, IEEE Transactions on Visualization and Computer Graphics.

[16]  Thies Pfeiffer,et al.  Advantages of eye-gaze over head-gaze-based selection in virtual and augmented reality under varying field of views , 2018, COGAIN@ETRA.

[17]  Joseph J. LaViola,et al.  Sharing gaze rays for visual target identification tasks in collaborative augmented reality , 2020, Journal on Multimodal User Interfaces.

[18]  Hai-Ning Liang,et al.  Pointing and Selection Methods for Text Entry in Augmented Reality Head Mounted Displays , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[19]  Mark Billinghurst,et al.  The Effect of View Independence in a Collaborative AR System , 2015, Computer Supported Cooperative Work (CSCW).

[20]  Bo Li,et al.  Multi-view VR system for co-located multidisciplinary collaboration and its application in ergonomic design , 2017, VRST.

[21]  Keita Higuchi,et al.  Can Eye Help You?: Effects of Visualizing Eye Fixations on Remote Collaboration Scenarios for Physical Tasks , 2016, CHI.

[22]  Hirokazu Kato,et al.  Real world teleconferencing , 1999, CHI Extended Abstracts.

[23]  John Paulin Hansen,et al.  Gaze typing compared with input by head and hand , 2004, ETRA.

[24]  Hannes Kaufmann,et al.  Collaborative Augmented Reality in Education , 2002 .

[25]  Xiang Li,et al.  Exploration of Hands-free Text Entry Techniques For Virtual Reality , 2020, 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[26]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[27]  Mark Billinghurst,et al.  CoVAR: a collaborative virtual and augmented reality system for remote collaboration , 2017, SIGGRAPH ASIA Emerging Technologies.

[28]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[29]  Robert E. Kraut,et al.  Using Visual Information for Grounding and Awareness in Collaborative Tasks , 2012, Hum. Comput. Interact..

[30]  Hiroshi Ishii,et al.  Iterative design of seamless collaboration media , 1994, CACM.

[31]  Jing Yang,et al.  A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing , 2020, CHI.

[32]  Weidong Huang,et al.  Sharing hand gesture and sketch cues in remote collaboration , 2019, J. Vis. Commun. Image Represent..

[33]  J. Volkert,et al.  A Toolbox Supporting Collaboration in Networked Virtual Environments , 2005, International Conference on Computational Science.

[34]  Barrett Ens,et al.  Modeling endpoint distribution of pointing selection tasks in virtual reality environments , 2019, ACM Trans. Graph..

[35]  Hirokazu Kato,et al.  Collaborative Mixed Reality , 1999 .

[36]  Michael Gervautz,et al.  Collaborative gaming in augmented reality , 1998, VRST '98.

[37]  Steven K. Feiner,et al.  Virtual Replicas for Remote Assistance in Virtual and Augmented Reality , 2015, UIST.

[38]  William Buxton,et al.  Mediaspace - Meaningspace - Meetingspace , 2009, Media Space 20+ Years of Mediated Life.

[39]  Thierry Duval,et al.  Improving awareness for 3D virtual collaboration by embedding the features of users’ physical environments and by augmenting interaction tools with cognitive feedback cues , 2014, Journal on Multimodal User Interfaces.

[40]  Jérôme Royan,et al.  Collaborators awareness for user cohabitation in co-located collaborative virtual environments , 2017, VRST.

[41]  Bruno Arnaldi,et al.  Scenario sharing in a collaborative virtual environment for training , 2008, VRST '08.

[42]  Mark Billinghurst,et al.  Study of augmented gesture communication cues and view sharing in remote collaboration , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[43]  Hui Huang.  SIGGRAPH ASIA 2016 Virtual Reality meets Physical Reality: Modelling and Simulating Virtual Humans and Environments , 2016, SIGGRAPH ASIA Virtual Reality meets Physical Reality.

[44]  Robert W. Lindeman,et al.  Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration , 2018, CHI.

[45]  Robert E. Kraut,et al.  Collaboration in performance of physical tasks: effects on outcomes and communication , 1996, CSCW '96.

[46]  Ka Lok Man,et al.  Collaborative behavior, performance and engagement with visual analytics tasks using mobile devices , 2020, Hum. centric Comput. Inf. Sci..

[47]  Ivan Poupyrev,et al.  Experiments with Face-To-Face Collaborative AR Interfaces , 2002, Virtual Reality.

[48]  Stephanie D. Teasley,et al.  The Construction of Shared Knowledge in Collaborative Problem Solving , 1995 .

[49]  Weidong Huang,et al.  HandsInTouch: sharing gestures in remote collaboration , 2018, OZCHI.

[50]  Henry Been-Lirn Duh,et al.  Augmented 3D hands: a gesture-based mixed reality system for distributed collaboration , 2017, Journal on Multimodal User Interfaces.

[51]  Harri Siirtola,et al.  Eye gaze and head gaze in collaborative games , 2019, ETRA.

[52]  Arindam Dey,et al.  Improving Collaboration in Augmented Video Conference using Mutually Shared Gaze , 2017, ICAT-EGVE.

[53]  Woontack Woo,et al.  Evaluating the Combination of Visual Communication Cues for HMD-based Mixed Reality Remote Collaboration , 2019, CHI.

[54]  Jonathan Grudin,et al.  Integration of interpersonal space and shared workspace: ClearBoard design and experiments , 1993, TOIS.

[55]  Gábor Sörös,et al.  The effects of spatial auditory and visual cues on mixed reality remote collaboration , 2020, Journal on Multimodal User Interfaces.

[56]  S. Harrison.  Media Space 20+ Years of Mediated Life , 2009, Computer Supported Cooperative Work.

[57]  Arindam Dey,et al.  The Effects of Sharing Awareness Cues in Collaborative Mixed Reality , 2019, Front. Robot. AI.

[58]  Hirotake Yamazoe,et al.  Synchronized AR environment for multiple users using animation markers , 2014, VRST '14.

[59]  Mark Billinghurst,et al.  The Effect of Collaboration Styles and View Independence on Video-Mediated Remote Collaboration , 2018, Computer Supported Cooperative Work (CSCW).

[60]  Hideaki Kuzuoka,et al.  WACL: supporting telecommunications using - wearable active camera with laser pointer , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[61]  Daniel Mendes,et al.  Warping Deixis: Distorting Gestures to Enhance Collaboration , 2019, CHI.