A Participatory Method for Multidimensional Analysis and Proposing Improvements in Collaborative Systems

The difficulty of evaluating collaborative systems is a historical challenge, since the multiple dimensions of collaboration are considered success factors for the system, and these factors go beyond individual interaction. Adopting the fundamentals of Participatory Design, this paper presents a multidimensional method for evaluating and proposing improvements on collaborative systems. The method involves three phases: (1) a presential briefing phase; (2) a non-presential observation phase; and (3) a Future Workshop phase. Instances of the proposed method seek to involve users as co-designers, making informed and relevant decisions about the future of a system which they could learn and use. This paper also presents the results of a preliminary evaluation, which consists in an instance of the method: evaluating and proposing improvements on a groupware prototype. The instance is compared to a traditional groupware evaluation method, exposing that the instance, not only identified similar collaboration-related issues, but also identified issues that were not identified by the inspection. During the experiment, participants informed us about issues experienced while collaborating and how they were perceived, and elicited possible futures in which the system would have more value to their objectives.

[1]  Mary Beth Rosson,et al.  Evaluating computer-supported cooperative work: models and frameworks , 2004, CSCW.

[2]  Douglas Schuler,et al.  Participatory Design: Principles and Practices , 1993 .

[3]  Peter B. Weinstein,et al.  In the Loop: The Organization of Team-Based Communication in a Patient-Centered Clinical Collaboration System , 2016, JMIR human factors.

[4]  Carl Gutwin,et al.  Empirical development of a heuristic evaluation methodology for shared workspace groupware , 2002, CSCW '02.

[5]  Carl Gutwin,et al.  Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration , 2003, TCHI.

[6]  Pedro Antunes,et al.  Structuring dimensions for collaborative systems evaluation , 2012, CSUR.

[7]  Jonathan Grudin,et al.  Computer-supported cooperative work: history and focus , 1994, Computer.

[8]  Myriam Lewkowicz,et al.  Design and evaluation of activity model-based groupware: methodological issues , 2005, 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise (WETICE'05).

[9]  Jonathan Grudin,et al.  Groupware and social dynamics: eight challenges for developers , 1994, CACM.

[10]  Massimo Maresca,et al.  Always-on distributed spreadsheet mashups , 2010, Mashups '09/'10.

[11]  Jonathan Grudin,et al.  Why CSCW applications fail: problems in the design and evaluationof organizational interfaces , 1988, CSCW '88.

[12]  Sandeep Purao,et al.  Situating evaluation in scenarios of use , 2004, CSCW.

[13]  Michael J. Muller,et al.  Participatory design , 1993, CACM.

[14]  Robert Jungk,et al.  Future Workshops: How to Create Desirable Futures , 1996 .

[15]  Volker Wulf,et al.  How to make software softer—designing tailorable applications , 1997, DIS '97.

[16]  Jonathan Grudin,et al.  Taxonomy and Theory in Computer Supported Cooperative Work , 2012 .

[17]  Clay Spinuzzi,et al.  A Scandinavian challenge, a US response: methodological assumptions in Scandinavian and US prototyping approaches , 2002, SIGDOC '02.

[18]  Natalia Sales Santos,et al.  An Overview of Evaluation Methods for Collaborative Systems , 2012, 2012 Brazilian Symposium on Collaborative Systems.

[19]  Raquel Oliveira Prates,et al.  Caracterização das adaptações em métodos de avaliação para aplicações colaborativas , 2012, IHC.

[20]  Mariano Pimentel,et al.  Modelo 3C de Colaboração para o desenvolvimento de Sistemas Colaborativos , 2006 .

[21]  Tom Dayton,et al.  Participatory Practices in the Software Lifecycle , 1997 .

[22]  Françoise Détienne,et al.  An approach to assess the quality of collaboration in technology-mediated design situations , 2009, ECCE.