Supporting Virtual Collaboration in Spatial Design Tasks: Are Surrogate or Natural Gestures More Effective?

Hand gestures offer a rich source of information for communication of spatial concepts. However, gestures are often lost when people work together in virtual space. Gestures may be conveyed in virtual spaces through many approaches, for example by drawing circles and arrows around objects on a shared screen, or using natural, real-time videos of a distant collaborator's hands or body. The first is an example of “surrogate” gestures and the second of “natural” gestures. The first is relatively inexpensive but requires users take some effort to draw and erase temporary symbols; the second is more costly but also more expressive and natural. This work will: 1) provide a brief, high-level framework characterizing tools that support virtual gestures; and 2) present an experimental study comparing the cognitive effort required when using sketched surrogate gestures versus natural gestures in a virtual mechanical design tasks. The larger goal is to understand whether the additional cost to support natural gestures is warranted, thus informing the design of future virtual collaboration tools. We recruited 24 participants and asked them to collaborate using three different approaches: face-to-face (as a baseline), virtual-sketching (which allows users to make gestural sketches on their drawings), and virtual-sketching with natural hand videos projected on the drawing surface. We found that users' cognitive workload was significantly reduced when natural hand videos were added to a virtual-sketching environment. These results suggest that the ease and expressivity of natural gestures provide more benefits than simpler surrogate gestures for virtual mechanical design tasks.

[1]  Susan R. Fussell,et al.  Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks , 2004, Hum. Comput. Interact..

[2]  E. McDonough,et al.  An investigation of the use of global, virtual, and colocated new product development teams , 2001 .

[3]  C. Handy.  Trust and the virtual organization , 1999 .

[4]  Sara A. Bly,et al.  A use of drawing surfaces in different collaborative settings , 1988, CSCW '88.

[5]  Robert E. Kraut,et al.  Visual Information as a Conversational Resource in Collaborative Physical Tasks , 2003, Hum. Comput. Interact..

[6]  Sara A. Bly,et al.  Managing a trois: a study of a multi-user drawing tool in distributed design work , 1991, CHI.

[7]  Ned Kock,et al.  Trust and leadership in virtual teamwork: A media naturalness perspective , 2004 .

[8]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[9]  Hideaki Kuzuoka,et al.  GestureMan: a mobile robot that embodies a remote instructor's actions , 2000, CSCW '00.

[10]  Mark Pendergast,et al.  Gesturing through cursors: implementing multiple pointers in group support systems , 1993, [1993] Proceedings of the Twenty-sixth Hawaii International Conference on System Sciences.

[11]  Mcdp Mathieu Weggeman,et al.  The Effect of IT and Co-location on Knowledge Dissemination , 2007 .

[12]  T. Allen.  Managing the flow of technology , 1977 .

[13]  David Stanley Kirk,et al.  Turn it this way : remote gesturing in video-mediated communication , 2007 .

[14]  John C. Tang,et al.  A framework for understanding the workspace activity of design teams , 1988, CSCW '88.

[15]  M. Maznevski,et al.  Bridging Space Over Time: Global Virtual Team Dynamics and Effectiveness , 2000 .

[16]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[17]  Caroline C. Hayes,et al.  What Skills Will Future Engineering Graduates Need in Global Organizations , 2007 .

[18]  R. Krauss,et al.  PSYCHOLOGICAL SCIENCE Research Article GESTURE, SPEECH, AND LEXICAL ACCESS: The Role of Lexical Movements in Speech Production , 2022 .

[19]  Xilin Chen,et al.  Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks , 2003, ICMI '03.

[20]  David Knoke,et al.  “SKONK WORKS HERE”: ACTIVATING NETWORK SOCIAL CAPITAL IN COMPLEX COLLABORATIONS , 2004 .

[21]  Martha W. Alibali,et al.  Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information , 2005, Spatial Cogn. Comput..

[22]  Hiroshi Ishii,et al.  MirrorFugue: communicating hand gesture in remote piano collaboration , 2010, TEI.

[23]  Judith S. Olson,et al.  Distance Matters , 2000, Hum. Comput. Interact..

[24]  M. Alibali,et al.  Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen , 2001 .

[25]  Tom Rodden,et al.  Turn it this way: grounding collaborative action with remote gestures , 2007, CHI.

[26]  Paul Dourish,et al.  The human infrastructure of cyberinfrastructure , 2006, CSCW '06.

[27]  Gloria Mark,et al.  Extreme collaboration , 2002, CACM.

[28]  S. Bly,et al.  Commune: a shared drawing surface , 1990, COCS '90.

[29]  Richard L. Daft,et al.  Organizational information requirements, media richness and structural design , 1986 .

[30]  Robert E. Kraut,et al.  Patterns of contact and communication in scientific research collaboration , 1990, CSCW '88.

[31]  Christina Haas,et al.  Does the Medium Make a Difference? Two Studies of Writing With Pen and Paper and With Computers , 1989, Hum. Comput. Interact..

[32]  Caroline Joyce Cornelius.  Development and evaluation of gesture rich collaborative drawing tools. , 2011 .

[33]  K. Emmorey,et al.  Gesture, Thought, and Spatial Language , 2001 .

[34]  Blake Ives,et al.  Virtual teams: a review of current literature and directions for future research , 2004, DATB.