Specification for Testing

The success of model-based testing, in automating the testing of an implementation given its state-based (or model-based) specification, raises the question of how best the specification can be tweaked in order to facilitate that process. This paper discusses several answers. Motivated by an example from web-based systems, and taking account of the restriction imposed by the testing interface, it considers both functional and non-functional properties. The former include laws, implicit system invariants and other consistency conditions whilst the latter include security leaks. It concludes that because of the importance of the link between specification and implementation in the testing process, there is a trade-off between genuinely useful testing information and the incorporation of some degree of information about the link, not normally regarded as part of the specification.

[1]  Padmanabhan Krishnan.  Uniform descriptions for model based testing , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[2]  Jonathan P. Bowen,et al.  ZUM '95: The Z Formal Specification Notation , 1995, Lecture Notes in Computer Science.

[3]  Thomas Santen,et al.  Automating Test Case Generation from Z Specifications with Isabelle , 1997, ZUM.

[4]  Philip Alan Stocks,et al.  Applying formal methods to software testing , 1993 .

[5]  Bernhard K. Aichernig,et al.  Test-Design through Abstraction - A Systematic Approach Based on the Refinement Calculus , 2001, J. Univers. Comput. Sci..

[6]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.

[7]  Bruno Legeard,et al.  Automated Boundary Testing from Z and B , 2002, FME.

[8]  Hans-Martin Hörcher,et al.  Improving Software Tests Using Z Specifications , 1995, ZUM.

[9]  Mark Utting.  The Role of Model-Based Testing , 2005, VSTTE.

[10]  Erich Mikk,et al.  Compilation of Z Specifications into C for Automatic Test Result Evaluation , 1995, ZUM.

[11]  Jan Peleska,et al.  Using formal specifications to support software testing , 1995, Software Quality Journal.

[12]  James A. Whittaker,et al.  Model‐Based Software Testing , 2002 .

[13]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[14]  Bernhard K. Aichernig,et al.  Automatic Test Case Generation for RAISE , 2004 .

[15]  T. Kanade.  Model-Based Testing of Reactive Systems , 2005 .

[16]  Robert Meersman,et al.  On the Move to Meaningful Internet Systems 2004: CoopIS, DOA, and ODBASE , 2004, Lecture Notes in Computer Science.

[17]  Anne Elisabeth Haxthausen,et al.  The Raise Specification Language , 1992 .

[18]  Ian J. Hayes,et al.  Specification directed module testing , 1986, IEEE Transactions on Software Engineering.

[19]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems: Advanced Lectures (Lecture Notes in Computer Science) , 2005 .

[20]  Jim Woodcock,et al.  FME '93: Industrial-Strength Formal Methods , 1993, Lecture Notes in Computer Science.

[21]  Jonathan P. Bowen,et al.  ZUM '97: The Z Formal Specification Notation , 1997, Lecture Notes in Computer Science.

[22]  Miquel Bertran,et al.  FME '94: Industrial Benefit of Formal Methods , 1994, Lecture Notes in Computer Science.

[23]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[24]  John J. Marciniak,et al.  Encyclopedia of Software Engineering , 1994, Encyclopedia of Software Engineering.

[25]  Roger Duke,et al.  Formal Object Oriented Specification Using Object-Z , 2000 .

[26]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[27]  Robert V. Binder,et al.  Design for testability in object-oriented systems , 1994, CACM.