CAMELOT: technology focused testing of CSCW applications

In this paper we describe CAMELOT, a technology-focused methodology for testing computer supported cooperative work (CSCW) software. Unlike previous CSCW evaluation techniques examining social issues, CAMELOT is intended for use by application developers, user interface specialists, performance engineers, and quality assurance personnel. The method divides an evaluation into two stages: single user and multi-user. The single user stage is subdivided into general computing and human-computer interaction testing. The multi-user stage is decomposed into distributed computing and human-human interaction testing. The methodology provides a detailed, codified, checklist of testing techniques for each stage. CAMELOT was applied to a conventionally tested, mature CSCW application and uncovered over two dozen problems.

[1]  B. Schneirdeman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[2]  James D. Hollan,et al.  Strategic directions in human-computer interaction , 1996, CSUR.

[3]  Tore Urnes,et al.  Tools for Implementing Groupware: Survey and Evaluation , 1994 .

[4]  Hidehiko Okada,et al.  GUITESTER: A Log-Based Usability Testing Tool for Graphical Usser Interfaces , 1999 .

[5]  Yvonne Rogers,et al.  PETRA: Participatory Evaluation Through Redesign and Analysis , 1995, Interact. Comput..

[6]  D. J. Robson,et al.  Applying formal specification and functional testing to graphical user interfaces , 1991, [1991] Proceedings, Advanced Computer Technology, Reliable Systems and Applications.

[7]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[8]  Clarence A. Ellis,et al.  Groupware: some issues and experiences , 1991, CACM.

[9]  Lynette Hirschman,et al.  Scenario-based evaluation of loosely-integrated collaborative systems , 2000, CHI Extended Abstracts.

[10]  Ramez Elmasri,et al.  Fundamentals of Database Systems , 1989 .

[11]  Henk Sol,et al.  Proceedings of the 54th Hawaii International Conference on System Sciences , 1997, HICSS 2015.

[12]  Ephraim P. Glinert,et al.  Exploring Collaborative Learning in Rensselaer's Classroom-in-the-Round , 1998 .

[13]  Carl Gutwin,et al.  Supporting awareness of others in groupware , 1996, CHI Conference Companion.

[14]  Ephraim P. Glinert,et al.  A testing methodology and architecture for computer supported cooperative work software , 2000 .

[15]  Prasun Dewan,et al.  Access control for collaborative environments , 1992, CSCW '92.

[16]  Saul Greenberg,et al.  GROUPWARE TOOLKITS FOR SYNCHRONOUS WORK , 1996 .

[17]  Jeff Sokolov.  Methodologies for evaluation of collaborative systems , 1999, SIGG.

[18]  Prasun Dewan,et al.  A high-level and flexible framework for implementing multiuser user interfaces , 1992, TOIS.

[19]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[20]  Thomas J. Ostrand,et al.  A visual test development environment for GUI systems , 1998, ISSTA.

[21]  Lee J. White.  Regression testing of GUI event interactions , 1996, 1996 Proceedings of International Conference on Software Maintenance.