Physicality As an Anchor for Coordination: Examining Collocated Collaboration in Physical and Mobile Augmented Reality Settings

Design and co-creation activities around 3D artifacts often require close collocated coordination between multiple users. Augmented reality (AR) technology can support collocated work enabling users to flexibly work with digital objects while still being able to use the physical space for coordination. With most of current research focusing on remote AR collaboration, less is known about collocated collaboration in AR, particularly in relation to interpersonal dynamics between the collocated collaborators. Our study aims at understanding how shared augmented reality facilitated by mobile devices (mobile augmented reality or MAR) affects collocated users' coordination. We compare the coordination behaviors that emerged in a MAR setting with those in a comparable fully physical setting by simulating the same task -of the shared physical dimension for participants' ability to coordinate in the context of collaborative co-creation. Namely, participants working in a fully physical setting were better able to leverage the work artifact itself for their coordination needs, working in a mode that we term artifact-oriented coordination. Conversely, participants collaborating around an AR artifact leveraged the shared physical workspace for their coordination needs, working in what we refer to as space-oriented coordination. We discuss implications for a AR-based collaboration and propose directions for designers of AR tools.

[1]  Steve Benford,et al.  Understanding and constructing shared spaces with mixed-reality boundaries , 1998, TCHI.

[2]  Mark Billinghurst,et al.  Face to face collaborative AR on mobile phones , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[3]  Françoise Détienne,et al.  Multimodal collaborative activity among architectural designers using an augmented desktop at distance or in collocation , 2008, ECCE '08.

[4]  Carl Gutwin,et al.  A Descriptive Framework of Workspace Awareness for Real-Time Groupware , 2002, Computer Supported Cooperative Work (CSCW).

[5]  M. Sheelagh T. Carpendale,et al.  Territoriality in collaborative tabletop workspaces , 2004, CSCW.

[6]  Roman Rädle,et al.  When Tablets meet Tabletops: The Effect of Tabletop Size on Around-the-Table Collaboration with Personal Tablets , 2016, CHI.

[7]  Susan P. Besemer Creative Product Analysis Matrix: Testing the Model Structure and a Comparison Among Products--Three Novel Chairs , 1998 .

[8]  Armin Hammand,et al.  Telegeoinformatics: Location-based Computing and Services , 2004 .

[9]  J. Baggs Development of an instrument to measure collaboration and satisfaction about care decisions. , 1994, Journal of advanced nursing.

[10]  Mi Jeong Kim,et al.  Technological advancements in synchronous collaboration: The effect of 3D virtual worlds and tangible user interfaces on architectural design , 2011 .

[11]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[12]  Sean Rintel,et al.  VROOM: Virtual Robot Overlay for Online Meetings , 2020, CHI Extended Abstracts.

[13]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[14]  Lei Gao,et al.  Wearable RemoteFusion: A Mixed Reality Remote Collaboration System with Local Eye Gaze and Remote Hand Gesture Sharing , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[15]  David Kirsh,et al.  Embodied cognition and the magical future of interaction design , 2013, TCHI.

[16]  Cecilia E. Ford Collaborative Construction of Task Activity: Coordinating Multiple Resources in a High School Physics Lab , 1999 .

[17]  Kevin Wong,et al.  Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts , 2007, DIMEA 2007.

[18]  Anthony Tang,et al.  Scale Impacts Elicited Gestures for Manipulating Holograms: Implications for AR Gesture Design , 2018, Conference on Designing Interactive Systems.

[19]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[20]  Yvonne Rogers,et al.  Actions speak loudly with words: unpacking collaboration around the table , 2009, ITS '09.

[21]  David Birchfield,et al.  Collaborative Embodied Learning in Mixed Reality Motion-Capture Environments: Two Science Studies , 2014 .

[22]  Duncan Sanderson,et al.  Coordinating joint design work: the role of communication and artefacts , 1998 .

[23]  Bertrand Schneider,et al.  What Can We Learn from Augmented Reality (AR)?: Benefits and Drawbacks of AR for Inquiry-based Learning of Physics , 2019, CHI.

[24]  Morten Fjeld,et al.  BUILD-IT: An Intuitive Design Tool Based on Direct Object Manipulation , 1997, Gesture Workshop.

[25]  Dieter Schmalstieg,et al.  “Studierstube”: An environment for collaboration in augmented reality , 1998, Virtual Reality.

[26]  Jan O. Borchers,et al.  Tangible Awareness: How Tangibles on Tabletops Influence Awareness of Each Other's Actions , 2018, CHI.

[27]  Lucy Suchman,et al.  Understanding practice: video as a medium for reflection and design (excerpt) , 1992 .

[28]  Soh-Khim Ong,et al.  A mixed reality environment for collaborative product design and development , 2009 .

[29]  Stephan Lukosch,et al.  Collaboration in Augmented Reality , 2015, Computer Supported Cooperative Work (CSCW).

[30]  Clemens Nylandsted Klokmose,et al.  Hybrid Collaboration – Moving Beyond Purely Co-Located or Remote Collaboration , 2019 .

[31]  John C. Tang Findings from Observational Studies of Collaborative Work , 1991, Int. J. Man Mach. Stud..

[32]  Oded Nov,et al.  Virtual Objects in the Physical World: Relatedness and Psychological Ownership in Augmented Reality , 2019, CHI.

[33]  Tom Rodden,et al.  Turn it this way: grounding collaborative action with remote gestures , 2007, CHI.

[34]  Steven Houben,et al.  CollabAR – Investigating the Mediating Role of Mobile AR Interfaces on Co-Located Group Collaboration , 2020, CHI.

[35]  Hooman Hedayati,et al.  Improving Collocated Robot Teleoperation with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[36]  Susanne Schmidt,et al.  When Joy Matters: The Importance of Hedonic Stimulation in Collocated Collaboration with Large-Displays , 2009, INTERACT.

[37]  Carman Neustaedter,et al.  VideoArms: Embodiments for Mixed Presence Groupware , 2007 .

[38]  Martin Hachet,et al.  One Reality: Augmenting How the Physical World is Experienced by combining Multiple Mixed Reality Modalities , 2017, UIST.

[39]  J. Rekimoto,et al.  Transvision: a hand-held augmented reality system for collaborative design , 1996 .

[40]  Arindam Dey,et al.  [POSTER] CoVAR: Mixed-Platform Remote Collaborative Augmented and Virtual Realities System with Shared Collaboration Cues , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[41]  Pamela J. Hinds,et al.  The (Currently) Unique Advantages of Collocated Work , 2002 .

[42]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[43]  Yvonne Rogers,et al.  HuddleLamp: Spatially-Aware Mobile Displays for Ad-hoc Around-the-Table Collaboration , 2014, ITS '14.

[44]  Dieter Schmalstieg,et al.  Mobile collaborative augmented reality , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[45]  Andy Cockburn,et al.  User-defined gestures for augmented reality , 2013, INTERACT.

[46]  Raimund Dachselt,et al.  PaperLens: advanced magic lens interaction above the tabletop , 2009, ITS '09.

[47]  Mark Billinghurst,et al.  Revisiting collaboration through mixed reality: The evolution of groupware , 2019, Int. J. Hum. Comput. Stud..

[48]  Ram D. Sriram,et al.  CAIRO: a system for facilitating communication in a distributed collaborative engineering environment , 1996 .

[49]  Leon D. Segal Effects of checklist interface on non-verbal crew communications , 1994 .

[50]  Terrence Fernando,et al.  A Distributed Virtual Environment for Collaborative Engineering , 1998, Presence.

[51]  Wei Liu,et al.  Mixed reality classroom: learning from entertainment , 2007, DIMEA.

[52]  Roman Rädle,et al.  Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices , 2019, CHI.

[53]  Greg Welch,et al.  Table-top spatially-augmented realty: bringing physical models to life with projected imagery , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[54]  Meredith Ringel Morris,et al.  Search on surfaces: Exploring the potential of interactive tabletops for collaborative search tasks , 2010, Inf. Process. Manag..

[55]  Augusto Esteves,et al.  A Look at the Effects of Handheld and Projected Augmented-reality on a Collaborative Task , 2018, SUI.

[56]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[57]  Signe Pedersen Staging negotiation spaces: A co-design framework , 2020 .

[58]  Rainer Stiefelhagen,et al.  Multi-user Collaboration on Complex Data in Virtual and Augmented Reality , 2018, HCI.

[59]  Roman Rädle,et al.  Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load , 2016, CHI.

[60]  Louis M. Gomez,et al.  A collaborative medium for the support of conversational props , 1992, CSCW '92.

[61]  Hyocheol Ro,et al.  AR Pointer: Advanced Ray-Casting Interface Using Laser Pointer Metaphor for Object Manipulation in 3D Augmented Reality Environment , 2019, Applied Sciences.

[62]  Woontack Woo,et al.  Evaluating the Combination of Visual Communication Cues for HMD-based Mixed Reality Remote Collaboration , 2019, CHI.

[63]  Samir Benbelkacem,et al.  Collocated learning experience within collaborative augmented environment (anatomy course) , 2017, 2017 5th International Conference on Electrical Engineering - Boumerdes (ICEE-B).

[64]  Xiangyu Wang,et al.  Comparative Effectiveness of Mixed Reality-Based Virtual Environments in Collaborative Design , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[65]  Arindam Dey,et al.  The Effects of Sharing Awareness Cues in Collaborative Mixed Reality , 2019, Front. Robot. AI.

[66]  Mark Billinghurst,et al.  A Survey of Augmented Reality , 2015, Found. Trends Hum. Comput. Interact..

[67]  Wilhelm Hasselbring,et al.  Research perspective on supporting software engineering via physical 3D models , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[68]  Kristine Nagel,et al.  Understanding the design space of referencing in collaborative augmented reality environments , 2007, GI '07.

[69]  Elena Márquez Segura,et al.  Astaire: A Collaborative Mixed Reality Dance Game for Collocated Players , 2019, CHI PLAY.

[70]  Mark Billinghurst,et al.  Shared space: An augmented reality approach for computer supported collaborative work , 1998, Virtual Reality.

[71]  Ofer Arazy,et al.  Normative Tensions in Shared Augmented Reality , 2018, Proc. ACM Hum. Comput. Interact..

[72]  Xiangyu Wang and Phillip S. Dunston GROUPWARE CONCEPTS FOR AUGMENTED REALITY MEDIATED HUMAN-TO-HUMAN COLLABORATION , 2006 .

[73]  Bin Xu,et al.  Attention from Afar: Simulating the Gazes of Remote Participants in Hybrid Meetings , 2017, Conference on Designing Interactive Systems.

[74]  Yoshifumi Kitamura,et al.  GyroWand: IMU-based Raycasting for Augmented Reality Head-Mounted Displays , 2015, SUI.

[75]  Meredith Ringel Morris,et al.  Exploring the effects of group size and table size on interactions with tabletop shared-display groupware , 2004, CSCW.

[76]  Arne Jönsson,et al.  The importance of eye-contact for collaboration in AR systems , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[77]  Steven K. Feiner,et al.  Opportunistic Tangible User Interfaces for Augmented Reality , 2010, IEEE Transactions on Visualization and Computer Graphics.

[78]  Weimin Mou,et al.  Approaches to the Design and Measurement of Social and Information Awareness in Augmented Reality Systems , 2003 .

[79]  Uwe Jasnoch,et al.  Integration techniques for distributed visualization within a virtual prototyping environment , 1996, Electronic Imaging.

[80]  Kenton O'Hara,et al.  Hybrid Meetings in the Modern Workplace: Stories of Success and Failure , 2019, CRIWG/CollabTech.

[81]  G. R. Oldham,et al.  Employee Creativity: Personal and Contextual Factors at Work , 1996 .

[82]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[83]  Eva Brandt,et al.  How Tangible Mock-Ups Support Design Collaboration , 2005, Nordes 2005: In the Making.

[84]  M. Sheelagh T. Carpendale,et al.  Collaborative coupling over tabletop displays , 2006, CHI.

[85]  E. William East,et al.  A Proposed Virtual Design Review Environment , 1997 .

[86]  Weidong Huang,et al.  HandsInTouch: sharing gestures in remote collaboration , 2018, OZCHI.

[87]  Donald A. Norman,et al.  Things That Make Us Smart: Defending Human Attributes In The Age Of The Machine , 1993 .

[88]  Yoonjung Park,et al.  A dynamic depth-variable ray-casting interface for object manipulation in ar environments , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[89]  Victoria Bellotti,et al.  Walking away from the desktop computer: distributed collaboration and mobility in a product design team , 1996, CSCW '96.

[90]  P. McIlvenny Situated Order: Studies in the Social Organisation of Talk and Embodied Activities , 1995 .

[91]  Hao Tang,et al.  Integrating AR and VR for Mobile Remote Collaboration , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[92]  Martin Schmitz,et al.  Permulin: mixed-focus collaboration on multi-view tabletops , 2014, CHI.

[93]  James N. and Campbell Dace A. Davidson,et al.  Collaborative Design in Virtual Space - GreenSpace II: A Shared Environment for Architectural Design Review , 1996, ACADIA proceedings.

[94]  Pan Hui,et al.  Mobile Augmented Reality Survey: From Where We Are to Where We Go , 2017, IEEE Access.

[95]  Nobuyoshi Yabuki,et al.  Differences in spatial understanding between physical and virtual models , 2014 .

[96]  Carl Gutwin,et al.  Design for individuals, design for groups: tradeoffs between power and workspace awareness , 1998, CSCW '98.

[97]  Tobias Isenberg,et al.  Collaborative Work in Augmented Reality: A Survey , 2020, IEEE Transactions on Visualization and Computer Graphics.

[98]  Marilyn Jager Adams,et al.  Situation Awareness and the Cognitive Management of Complex Systems , 1995, Hum. Factors.

[99]  Hannah Murphy,et al.  Blocks: Collaborative and Persistent Augmented Reality Experiences , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[100]  Frank Biocca,et al.  Attention Issues in Spatial Information Systems: Directing Mobile Users' Visual Attention Using Augmented Reality , 2007, J. Manag. Inf. Syst..

[101]  Anoop Gupta,et al.  Communication behaviors of co-located users in collaborative AR interfaces , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[102]  Erik Lerdahl,et al.  Staging for Creative Collaboration in Design Teams , 2008 .

[103]  Xiangyu Wang,et al.  User perspectives on mixed reality tabletop visualization for face-to-face collaborative design review , 2008 .

[104]  BenfordSteve,et al.  Understanding and constructing shared spaces with mixed-reality boundaries , 1998 .