Improving co-presence with augmented visual communication cues for sharing experience through video conference

Video conferencing is becoming more widely used in areas other than face-to-face conversation, such as sharing real world experience with remote friends or family. In this paper we explore how adding augmented visual communication cues can improve the experience of sharing remote task space and collaborating together. We developed a prototype system that allows users to share live video view of their task space taken on a Head Mounted Display (HMD) or Handheld Display (HHD), and communicate through not only voice but also using augmented pointer or annotations drawn on the shared view. To explore the effect of having such an interface for remote collaboration, we conducted a user study comparing three video-conferencing conditions with different combination of communication cues: (1) voice only, (2) voice + pointer, and (3) voice + annotation. The participants used our remote collaboration system to share a parallel experience of puzzle solving in the user study, and we found that adding augmented visual cues significantly improved the sense of being together. The pointer was the most preferred additional cue by users for parallel experience, and there were different states of the users' behavior found in remote collaboration.

[1]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[2]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[3]  Katashi Nagao,et al.  The world through the computer: computer augmented interaction with real world environments , 1995, UIST '95.

[4]  Wa Wijnand IJsselsteijn,et al.  Staying in touch : social presence and connectedness through synchronous and asynchonous communication media , 2003 .

[5]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[6]  Takanori Maesako,et al.  HyperMirror: toward pleasant-to-use video mediated communication system , 1998, CSCW '98.

[7]  Mark Billinghurst,et al.  A Wizard of Oz study for an AR multimodal interface , 2008, ICMI '08.

[8]  Hideaki Kuzuoka,et al.  Spatial workspace collaboration: a SharedView video support system for remote collaboration capability , 1992, CHI.

[9]  Jun Rekimoto,et al.  JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation , 2014, AH.

[10]  Dieter Schmalstieg,et al.  Remote Collaboration Using Augmented Reality Videoconferencing , 2004, Graphics Interface.

[11]  Hideaki Kuzuoka,et al.  GestureCam: a video communication system for sympathetic remote collaboration , 1994, CSCW '94.

[12]  Hirokazu Kato,et al.  THE EFFECT OF SPATIAL CUES IN AUGMENTED REALITY VIDEO CONFERENCING , 2001 .

[13]  Nicole Fruehauf.  Flow The Psychology Of Optimal Experience , 2016 .

[14]  Kori Inkpen Quinn,et al.  Video playdate: toward free play across distance , 2010, CHI.

[15]  Mark Billinghurst,et al.  A wearable spatial conferencing space , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[16]  Hideaki Kuzuoka,et al.  WACL: supporting telecommunications using - wearable active camera with laser pointer , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[17]  Weidong Huang,et al.  Remote Tele-assistance System for Maintenance Operators in Mines , 2011 .

[18]  Tobias Höllerer,et al.  Integrating the physical environment into mobile remote collaboration , 2012, Mobile HCI.

[19]  Tom Rodden,et al.  Turn it this way: grounding collaborative action with remote gestures , 2007, CHI.

[20]  Hirokazu Kato,et al.  Collaborative augmented reality , 2002, CACM.

[21]  Carman Neustaedter,et al.  Interpersonal awareness in the domestic realm , 2006, OZCHI.

[22]  Sungjae Hwang,et al.  Chili: viewpoint control and on-video drawing for mobile video calls , 2013, CHI Extended Abstracts.

[23]  John L. Arnott,et al.  Using spatial cues to improve videoconferencing , 1992, CHI.

[24]  Seungwon Kim,et al.  Comparing pointing and drawing for remote collaboration , 2013, ISMAR.

[25]  Darren Gergle,et al.  Effects of four computer-mediated communications channels on trust development , 2002, CHI.

[26]  David A. Forsyth,et al.  BeThere: 3D mobile collaboration with spatial input , 2013, CHI.

[27]  Susan R. Fussell,et al.  Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks , 2004, Hum. Comput. Interact..

[28]  John C. Tang,et al.  Experiences2Go: sharing kids' activities outside the home with remote family members , 2013, CSCW.

[29]  Hirokazu Kato,et al.  Real-time 3D interaction for augmented and virtual reality , 2002, SIGGRAPH '02.

[30]  Stuart Anderson,et al.  RemoteFusion: real time depth camera fusion for remote collaboration on physical tasks , 2013, VRCAI '13.

[31]  Hujun Bao,et al.  Robust monocular SLAM in dynamic environments , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[32]  Hirokazu Kato,et al.  AR-based Video-Mediated Communication: A Social Presence Enhancing Experience , 2012, 2012 14th Symposium on Virtual and Augmented Reality.

[33]  Yutaka Matsushita,et al.  Multiparty videoconferencing at virtual social distance: MAJIC design , 1994, CSCW '94.

[34]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).