An Automated Approach for Goal-driven, Specification-based Testing

This paper presents a specification-based approach that addresses several known challenges including false positives and domain knowledge errors. Our approach begins with a goal graph and plans. Source code is annotated with goals and events and precompiled to emit those at run time. Plans are automatically translated into a rule-based recognizer. An oracle is produced from the pre- and postconditions associated with the plan's goals. When the program is executed, goals and events are emitted and automatically tested against plans and oracles. The concept is demonstrated on a small example and a larger publicly available case study

[1]  Axel van Lamsweerde,et al.  Deriving operational software specifications from system goals , 2002, SIGSOFT '02/FSE-10.

[2]  James C. Corbett,et al.  Bandera: extracting finite-state models from Java source code , 2000, ICSE.

[3]  Alistair Cockburn,et al.  Structuring Use Cases with Goals , 2000 .

[4]  Alessandro Orso,et al.  Gamma system: continuous evolution of software after deployment , 2002, ISSTA '02.

[5]  Sebastián Uchitel,et al.  Fluent-based animation: exploiting the relation between goals and scenarios for requirements validation , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[6]  Axel van Lamsweerde,et al.  Inferring Declarative Requirements Specifications from Operational Scenarios , 1998, IEEE Trans. Software Eng..

[7]  Hojjat Adeli,et al.  Expert system shells , 1988 .

[8]  Thomas A. Alspaugh,et al.  Temporally Expressive Scenarios in ScenarioML , 2005 .

[9]  Thomas A. Alspaugh,et al.  Scenario-driven Specification-based Testing against Goals and Requirements , 2005 .

[10]  John Mylopoulos,et al.  Representing and Using Nonfunctional Requirements: A Process-Oriented Approach , 1992, IEEE Trans. Software Eng..

[11]  John B. Goodenough,et al.  Toward a theory of test data selection , 1975 .

[12]  Thomas A. Alspaugh,et al.  Using scenarios to support traceability , 2005, TEFSE '05.

[13]  Michael D. Ernst,et al.  An overview of JML tools and applications , 2003, International Journal on Software Tools for Technology Transfer.

[14]  Henry Muccini,et al.  Systematic Testing of Software Architectures in the C2 Style , 2004, FASE.

[15]  Mordechai Ben-Menachem,et al.  Writing effective use cases , 2001, SOEN.

[16]  David Coppit,et al.  On the Use of Specification-Based Assertions as Test Oracles , 2005, 29th Annual IEEE/NASA Software Engineering Workshop.

[17]  Alessandro Orso,et al.  Monitoring deployed software using software tomography , 2002, PASTE '02.

[18]  David S. Rosenblum.  A Practical Approach to Programming With Assertions , 1995, IEEE Trans. Software Eng..

[19]  Henry Muccini,et al.  Formal Methods in Testing Software Architectures , 2003, SFM.

[20]  Michal Young,et al.  Residual test coverage monitoring , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[21]  Leila Naslavsky.  Distributed expectation-driven residual testing , 2004, ICSE 2004.

[22]  Henry Muccini,et al.  Software architecture-based regression testing , 2006, J. Syst. Softw..

[23]  K. Barraclough.  Eclipse , 2006, BMJ : British Medical Journal.

[24]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[25]  Richard N. Taylor,et al.  Rethinking the taxonomy of fault detection techniques , 1989, ICSE '89.

[26]  Stephen Fickas,et al.  Goal-Directed Requirements Acquisition , 1993, Sci. Comput. Program..