An evaluation of software test environment architectures

Software test environments (STEs) provide a means of automating the test process and integrating testing tools to support required testing capabilities across the test process. Specifically, STEs may support test planning, test management, test measurement, test failure analysis, test development and test execution. The software architecture of an STE describes the allocation of the environment's functions to specific implementation structures. An STE's architecture can facilitate or impede modifications such as changes to processing algorithms, data representation or functionality. Performance and reusability are also subject to architecturally imposed constraints. Evaluation of an STE's architecture can provide insight into modifiability, extensibility, portability and reusability of the STE. This paper proposes a reference architecture for STEs. Its analytical value is demonstrated by using SAAM (Software Architectural Analysis Method) to compare three software test environments: PROTest II (PROLOG Test Environment, Version II), TAOS (Testing with Analysis and Oracle Support), and CITE (CONVEX Integrated Test Environment).

[1]  D. L. Parnas,et al.  On the criteria to be used in decomposing systems into modules , 1972, Software Pioneers.

[2]  Joseph P. Cavano,et al.  A framework for the measurement of software quality , 1978 .

[3]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[4]  T. Nomura.  Use of software engineering tools in Japan , 1987, ICSE '87.

[5]  Lori A. Clarke,et al.  TEAM: a support environment for testing, evaluation, and analysis , 1988, SDE 3.

[6]  David Gelperin,et al.  The growth of software testing , 1988, CACM.

[7]  James M. Bieman,et al.  Deriving Measures of Software Reuse in Object Oriented Systems , 1991, Formal Aspects of Measurement.

[8]  Debra J. Richardson,et al.  Specification-based test oracles for reactive systems , 1992, International Conference on Software Engineering.

[9]  David Garlan,et al.  Using tool abstraction to compose systems , 1992, Computer.

[10]  Lori A. Clarke,et al.  PLEIADES: an object management system for software engineering environments , 1993, SIGSOFT '93.

[11]  Mary Shaw,et al.  An Introduction to Software Architecture , 1993, Advances in Software Engineering and Knowledge Engineering.

[12]  Fevzi Belli,et al.  Implementation-based analysis and testing of Prolog programs , 1993, ISSTA '93.

[13]  Peter A. Vogel,et al.  An integrated general purpose automated test environment , 1993, ISSTA '93.

[14]  Stephen Norman,et al.  Software Testing Tools , 1993 .

[15]  Debra J. Richardson,et al.  TAOS: Testing with Analysis and Oracle Support , 1994, ISSTA '94.

[16]  Leonard J. Bass,et al.  SAAM: a method for analyzing the properties of software architectures , 1994, Proceedings of 16th International Conference on Software Engineering.