Continuous Coordination Tools and their Evaluation

This chapter discusses a set of co-ordination tools (the Continuous Co-ordination (CC) tool suite that includes Ariadne, Workspace Activity Viewer (WAV), Lighthouse, Palantir, and YANCEES) and details of our evaluation framework for these tools. Specifically, we discuss how we assessed the usefulness and the usability of these tools within the context of a predefined evaluation framework called DESMET . For example, for visualization tools we evaluated the suitability of the level of abstraction and the mode of displaying information of each tool. Whereas for an infrastructure tool we evaluate the effort required to implement co-ordination tools based on the given tool. We conclude with pointers on factors to consider when evaluating co-ordination tools in general.

[1]  Louise Barkhuus,et al.  From Mice to Men - 24 Years of Evaluation in CHI , 2007, CHI 2007.

[2]  David F. Redmiles,et al.  A Survey of Versatility for Publish/Subscribe Infrastructures , 2005 .

[3]  Giorgio De Michelis,et al.  Directional interaction with large displays using mobile phones , 2006, Fourth Annual IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOMW'06).

[4]  Thomas R. G. Green,et al.  Cognitive dimensions of notations , 1990 .

[5]  Ping Chen,et al.  Lighthouse: coordination through emerging design , 2006, eclipse '06.

[6]  Paul Dourish,et al.  Seeking the source: software source code as a social and technical artifact , 2005, GROUP.

[7]  André van der Hoek,et al.  Palantir: raising awareness among configuration management workspaces , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[8]  David F. Redmiles,et al.  Analyzing a socio-technical visualization tool using usability inspection methods , 2008, 2008 IEEE Symposium on Visual Languages and Human-Centric Computing.

[9]  Tore Dybå,et al.  The Future of Empirical Methods in Software Engineering Research , 2007, Future of Software Engineering (FOSE '07).

[10]  David Morera.  COTS Evaluation Using Desmet Methodology & Analytic Hierarchy Process (AHP) , 2002, PROFES.

[11]  Michel Beaudouin-Lafon,et al.  Computer supported co-operative work , 1999 .

[12]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .

[13]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[14]  Jonathan Grudin,et al.  Groupware and social dynamics: eight challenges for developers , 1994, CACM.

[15]  Xiaojun Bi,et al.  uPen: a smart pen-liked device for facilitating interaction on large displays , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[16]  Michele Lanza,et al.  Reverse Engineering Super-Repositories , 2007, 14th Working Conference on Reverse Engineering (WCRE 2007).

[17]  André van der Hoek,et al.  Towards supporting awareness of indirect conflicts across software configuration management workspaces , 2007, ASE '07.

[18]  Jean Scholtz,et al.  Evaluation for collaborative systems , 1999, CSUR.

[19]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[20]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[21]  David S. Rosenblum,et al.  Design and evaluation of a wide-area event notification service , 2001, TOCS.

[22]  Jonathan Grudin,et al.  Why CSCW Applications Fail: Problems in the Design and Evaluation of Organization of Organizational Interfaces. , 1988 .

[23]  Cleidson R. B. de Souza,et al.  Lessons Learned Using Notification Servers To Support Application Awareness , 2003 .

[24]  Kevin Crowston,et al.  The interdisciplinary study of coordination , 1994, CSUR.

[25]  Keith Andrews,et al.  Evaluating information visualisations , 2006, BELIV '06.

[26]  Elizabeth D. Mynatt,et al.  Secrets to success and fatal flaws: the design of large-display groupware , 2006, IEEE Computer Graphics and Applications.

[27]  David F. Redmiles,et al.  Supporting collaborative software development through the visualization of socio-technical dependencies , 2007, GROUP.

[28]  David F. Redmiles,et al.  On the relationship between software dependencies and coordination: field studies and tool support , 2005 .

[29]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[30]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994, Structural analysis in the social sciences.

[31]  Paul Dourish,et al.  "Breaking the code", moving between private and public work in collaborative software development , 2003, GROUP.

[32]  Paul Dourish,et al.  In the eye of the beholder: A visualization-based approach to information system security , 2005, Int. J. Hum. Comput. Stud..

[33]  Li-Te Cheng,et al.  Shared waypoints and social tagging to support collaboration in software development , 2006, CSCW '06.

[34]  Henrik Hedberg,et al.  A preliminary evaluation of software inspection tools, with the DESMET method , 2005, Fifth International Conference on Quality Software (QSIC'05).

[35]  C. Amrit.  Coordination in Software Development: The problem of Task Allocation , 2005 .

[36]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[37]  Jonathan Grudin,et al.  Why CSCW applications fail: problems in the design and evaluationof organizational interfaces , 1988, CSCW '88.

[38]  R. Magnus,et al.  The learning way:evaluating co-operative systems , 1999 .

[39]  David F. Redmiles,et al.  An approach to large-scale collection of application usage data over the internet , 1998, Proceedings of the 20th International Conference on Software Engineering.

[40]  Magnus Ramage,et al.  The learning way : evaluating co-operative systems , 1999 .

[41]  Carmelo Ardito,et al.  Systematic inspection of information visualization systems , 2006, BELIV '06.

[42]  David F. Redmiles,et al.  Striving for versatility in publish/subscribe infrastructures , 2005, SEM '05.

[43]  M. Sheelagh T. Carpendale,et al.  Heuristics for information visualization evaluation , 2006, BELIV '06.

[44]  M. Sheelagh T. Carpendale,et al.  Evaluating Information Visualizations , 2008, Information Visualization.

[45]  Barbara Ann Kitchenham,et al.  Evaluating software engineering methods and tool part 1: The evaluation context and evaluation methods , 1996, SOEN.

[46]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[47]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[48]  André van der Hoek,et al.  A Visualization for Software Project Awareness and Evolution , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[49]  David F. Redmiles,et al.  Creating an Infrastructure for Ubiquitous Awareness , 2001, INTERACT.

[50]  John F. Patterson,et al.  Notification servers for synchronous groupware , 1996, CSCW '96.

[51]  Hugh Whitemore,et al.  Breaking the Code , 1986 .

[52]  Li-Te Cheng,et al.  How a good software practice thwarts collaboration: the multiple roles of APIs in software development , 2004, SIGSOFT '04/FSE-12.

[53]  André van der Hoek,et al.  Continuous coordination a new paradigm to support globally distributed software development projects , 2007 .

[54]  James D. Herbsleb,et al.  Identification of coordination requirements: implications for the Design of collaboration and awareness tools , 2006, CSCW '06.

[55]  Kevin Crowston,et al.  A Coordination Theory Approach to Organizational Process Design , 1997 .

[56]  Wendy A. Kellogg,et al.  Task and social visualization in software development: evaluation of a prototype , 2007, CHI.

[57]  Chintan Amrit.  Coordination in software development: the problem of task allocation , 2005, ACM SIGSOFT Softw. Eng. Notes.

[58]  E. Tufte.  Beautiful Evidence , 2006 .

[59]  Paul A. Strooper,et al.  Evaluating software refactoring tool support , 2006, Australian Software Engineering Conference (ASWEC'06).

[60]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .