A laboratory method for studying activity awareness

Many failures in long-term collaboration occur because of a lack of activity awareness. Activity awareness is a broad concept that involves awareness of synchronous and asynchronous interactions over extended time periods. We describe a procedure to evaluate activity awareness and collaborative activities in a controlled setting. The activities used are modeled on real-world collaborations documented earlier in a field study. We developed an experimental method to study these activity awareness problems in the laboratory. Participants worked on a simulated long-term project in the laboratory over multiple experimental sessions with a confederate, who partially scripted activities and probes. We present evidence showing that this method represents a valid model of real collaboration, based on participants' active engagement, lively negotiation, and awareness difficulties. We found that having the ability to define, reproduce, and systematically manipulate collaborative situations allowed us to assess the effect of realistic conditions on activity awareness in remote collaboration.

[1]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[2]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[3]  M. Scriven.  The methodology of evaluation , 1966 .

[4]  Barbara Hayes-Roth,et al.  A Cognitive Model of Planning , 1979, Cogn. Sci..

[5]  Terry Winograd,et al.  Understanding computers and cognition - a new foundation for design , 1987 .

[6]  L. Suchman.  Plans and situated actions , 1987 .

[7]  Jonathan Grudin,et al.  Why CSCW Applications Fail: Problems in the Design and Evaluation of Organization of Organizational Interfaces. , 1988 .

[8]  J. Valacich,et al.  Effects of anonymity and evaluative tone on idea generation in computer-mediated groups , 1990 .

[9]  Kari Kuutti,et al.  Identifying potential CSCW applications by means of activity theory concepts: a case example , 1992, CSCW '92.

[10]  S. Easterbrook.  CSCW: Cooperation or Conflict? , 1993, Computer Supported Cooperative Work.

[11]  Steve Easterbrook,et al.  Cooperation or Conflict , 1993 .

[12]  Richard Bentley,et al.  Situated evaluation for cooperative systems , 1994, CSCW '94.

[13]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[14]  Jonathan Grudin,et al.  Human Computer Interaction: The Year 2000 and Beyond , 1995, HCI.

[15]  Ronald M. Baecker,et al.  Readings in human-computer interaction : toward the year 2000 , 1995 .

[16]  J. McGrath.  Methodology matters: doing research in the behavioral and social sciences , 1995 .

[17]  Eduardo Salas,et al.  Situation Awareness in Team Performance: Implications for Measurement and Training , 1995, Hum. Factors.

[18]  Carl Gutwin,et al.  Workspace awareness for groupware , 1996, CHI Conference Companion.

[19]  Andrew F. Monk,et al.  Inter-personal awareness and synchronization: assessing the value of communication technologies , 1996, Int. J. Hum. Comput. Stud..

[20]  Gloria Mark,et al.  Designing groupware for congruency in use , 1998, CSCW '98.

[21]  Magnus Ramage,et al.  The learning way : evaluating co-operative systems , 1999 .

[22]  John M. Carroll,et al.  Multi-faceted evaluation for complex, distributed activities , 1999, CSCL.

[23]  Holly Arrow,et al.  Small Groups as Complex Systems: Formation, Coordination, Development, and Adaptation , 2000 .

[24]  Carl Gutwin,et al.  The mechanics of collaboration: developing low cost usability evaluation methods for shared workspaces , 2000, Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000).

[25]  Ilze Zigurs,et al.  Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work , 2001 .

[26]  Carl Gutwin,et al.  A comparison of usage evaluation and inspection methods for assessing groupware usability , 2001, GROUP.

[27]  S. Mohammed,et al.  Team mental models in a team knowledge framework: expanding theory and measurement across disciplinary boundaries , 2001 .

[28]  Kevin F. Baker,et al.  Heuristic Evaluation of Shared Workspace Groupware based on the Mechanics of Collaboration , 2002 .

[29]  Gilbert Cockton,et al.  Sale must end: should discount methods be cleared off HCI's shelves? , 2002, INTR.

[30]  Carl Gutwin,et al.  Groupware walkthrough: adding context to groupware usability evaluation , 2002, CHI.

[31]  Carl Gutwin,et al.  Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration , 2003, TCHI.

[32]  Mary Beth Rosson,et al.  Notification and awareness: synchronizing task-oriented collaborative activity , 2003, Int. J. Hum. Comput. Stud..

[33]  Mary Beth Rosson,et al.  Classroom BRIDGE: using collaborative public and desktop timelines to support activity awareness , 2003, UIST '03.

[34]  Suzanne P. Weisband,et al.  Overcoming social awareness in computer-supported groups , 1993, Computer Supported Cooperative Work (CSCW).

[35]  Mary Beth Rosson,et al.  Evaluating computer-supported cooperative work: models and frameworks , 2004, CSCW.