Measures of collaboration in CSCW: Usability and collective measures in remote and co-located problem-solving

Since computers first came to be used in the society the role of the user has changed as has the research connected to it. Computer Supported Collaborative Work (CSCW) has focused on several users and to evaluate existing collaborative systems. However, no full method is yet proposed. Authors state that there is a lack in usability and social studies. The aim of this thesis has first been to find new measures for collaboration and second to test the validity of these measures on a set of experimental data. A literature review showed that good candidates for new measures were usability an collective measures. A careful observation of the experimental data set helped to find seven candidate measures: Speaking, Monitoring, Looking at task description, Sketching, Erasing, Laughing and Task process that were applied on an existing body of experimental data. The collection of experimental data stemmed from video taped observations on a within-group design experiment involving eighteen participants. Nine pairs of subjects solved dot-to-dot puzzle problems under three conditions (first independent variable), two of these conditions were remote (SkypeVideo and CollaBoard); one condition was co-located (Co-located). There were three tasks with varying difficulty (second independent variable). Monitoring and Speaking were examined in statistical tests and the five other measures in descriptive analysis. Statistical significant difference in means were found for tasks in Monitoring and Speaking. In Monitoring the difference was due to the difference in task time. No significance was seen for conditions which could be due to a high variation as seen in the box-plots or that the conditions were too similar when solving the specific dot-to-dot puzzle problem. The descriptive data showed similar results. The Task process measure showed that in the Co-located condition most focus was put on task-solving, in Skype Video the most focus was put on the whiteboard. In the CollaBoard condition Looking at task description had the most focus and the most conversation and Laughing was seen. Most Laughing was also seen in the simple and hard task. Also the most focus on the whiteboard was seen with the simple task. Most task-solving was done in the average task which implies that in an average task more focus can be put on solving the task than on other things. The measures found in this study should be investigated further as well as measures of CSCW usability and collectivity in general. A more usable CSCW system may save time, money, and our environment.

[1]  Carl Gutwin,et al.  Group awareness in distributed software development , 2004, CSCW.

[2]  Françoise Détienne,et al.  Assessing the "Quality of Collaboration" in Technology-Mediated Design Situations with Several Dimensions , 2009, INTERACT.

[3]  Julien Epps,et al.  A study of co-worker awareness in remote collaboration over a shared application , 2007, CHI Extended Abstracts.

[4]  Atul Prakash,et al.  DistView: support for building efficient collaborative applications using replicated objects , 1994, CSCW '94.

[5]  Susan R. Fussell,et al.  Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks , 2004, Hum. Comput. Interact..

[6]  Dirk Heylen,et al.  Collaborative Practices that Support Creativity in Design , 2009, ECSCW.

[7]  Paul Dourish,et al.  Portholes: supporting awareness in a distributed work group , 1992, CHI.

[8]  Carl Gutwin,et al.  Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration , 2001, EHCI.

[9]  David Kaufer,et al.  Computer support for distributed collaborative writing: defining parameters of interaction , 1994, CSCW '94.

[10]  Gayle J. Yaverbaum,et al.  Asynchronous Computer-mediated Communication versus Face-to-face Collaboration: Results on Student Learning, Quality and Satisfaction , 1999 .

[11]  Saul Greenberg,et al.  Supporting Awareness in Mixed Presence Groupware , 2005 .

[12]  David S. Kirk,et al.  Comparing remote gesture technologies for supporting collaborative physical tasks , 2006, CHI.

[13]  Susan R. Fussell,et al.  Assessing the value of a cursor pointing device for remote collaboration on physical tasks , 2003, CHI Extended Abstracts.

[14]  Tom Rodden,et al.  Ways of the Hands , 2005, ECSCW.

[15]  Xiaoyu Zhang,et al.  Quantifying the benefits of immersion for collaboration in virtual environments , 2005, VRST '05.

[16]  Gerrit C. van der Veer,et al.  Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems , 1993 .

[17]  Motoshi Saeki.  Communication, collaboration and cooperation in software development-how should we support group work in software development? , 1995, Proceedings 1995 Asia Pacific Software Engineering Conference.

[18]  Carl Gutwin,et al.  A Descriptive Framework of Workspace Awareness for Real-Time Groupware , 2002, Computer Supported Cooperative Work (CSCW).

[19]  Carl Gutwin,et al.  Artifact awareness through screen sharing for distributed groups , 2009, Int. J. Hum. Comput. Stud..

[20]  Sara A. Bly,et al.  A use of drawing surfaces in different collaborative settings , 1988, CSCW '88.

[21]  Wendy Ju,et al.  Thinking with Erasable Ink : Ad-hoc Whiteboard Use in Collaborative Design , 2006 .

[22]  Anastasia Bezerianos,et al.  Presence disparity in mixed presence collaboration , 2008, CHI Extended Abstracts.

[23]  Simon M. Kaplan,et al.  Flexible, active support for collaborative work with ConversationBuilder , 1992, CSCW '92.

[24]  Mary Beth Rosson,et al.  Evaluating computer-supported cooperative work: models and frameworks , 2004, CSCW.

[25]  Regan L. Mandryk,et al.  System Guidelines for Co-located, Collaborative Work on a Tabletop Display , 2003, ECSCW.

[26]  Chengzhi Peng,et al.  Survey of collaborative drawing support tools , 1992, Computer Supported Cooperative Work (CSCW).

[27]  Jonathan Grudin,et al.  Why CSCW Applications Fail: Problems in the Design and Evaluation of Organization of Organizational Interfaces. , 1988 .

[28]  John L. Bennett,et al.  Learning from user experience with groupware , 1990, CSCW '90.

[29]  David S. Kirk,et al.  The effects of remote gesturing on distance instruction , 2005, CSCL.

[30]  Joseph F. McCarthy,et al.  MusicFX: an arbiter of group preferences for computer supported collaborative workouts , 1998, CSCW '98.

[31]  John C. Tang,et al.  Videodraw: a video interface for collaborative drawing , 1991, TOIS.

[32]  Wanda J. Orlikowski,et al.  Learning from Notes: organizational issues in groupware implementation , 1992, CSCW '92.

[33]  Tek-Jin Nam,et al.  Collaborative 3D Workspace and Interaction Techniques for Synchronous Distributed Product Design Reviews , 2009 .

[34]  Masanori Sugimoto,et al.  Caretta: a system for supporting face-to-face collaboration by integrating personal and shared spaces , 2004, CHI.

[35]  Nuria Oliver,et al.  Gaze and Gestures in Telepresence: multimodality, embodiment, and roles of collaboration , 2010, ArXiv.

[36]  Walter Bender,et al.  The Impact of Increased Awareness While Face-to-Face , 2007, Hum. Comput. Interact..

[37]  Carman Neustaedter,et al.  VideoArms: Embodiments for Mixed Presence Groupware , 2007 .

[38]  Carl Gutwin,et al.  The mechanics of collaboration: developing low cost usability evaluation methods for shared workspaces , 2000, Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000).