Specification Centered Testing

This position paper discusses a framework for automating the testing of systems with stringent structural coverage requirements, for example, avionics systems. The framework covers testing of the model of the desired behavior as well as the resulting implementation. We use a formal model of the required software behavior as the central component of our testing strategy; we call this approach specification centered testing . We discuss how a model checker can be used to automatically generate complete test sequences that will provide arbitrary structural coverage of the requirements specification as well as the code implementing the requirements.

[1]  J. McDermid,et al.  Testing , Proof and Automation . An Integrated Approach , 2000 .

[2]  Mark Blackburn,et al.  Automatic generation of test vectors for SCR-style specifications , 1997, Proceedings of COMPASS '97: 12th Annual Conference on Computer Assurance.

[3]  Kenneth L. McMillan,et al.  Symbolic model checking , 1992 .

[4]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[5]  Michael W. Whalen,et al.  A formal semantics for RSML- e , 2000 .

[6]  John Rushby,et al.  The PVS Specification Language , 1993 .

[7]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[8]  Mats Per Erik Heimdahl,et al.  Specification-based prototyping for embedded systems , 1999, ESEC/FSE-7.

[9]  Matthew B. Dwyer,et al.  Bandera: extracting finite-state models from Java source code , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[10]  John Rushby,et al.  Model Checking and Other Ways of Automating Formal Methods , 1995 .

[11]  Sanjai Rayadurgam,et al.  Test-sequence generation from formal requirement models , 2001, Proceedings Sixth IEEE International Symposium on High Assurance Systems Engineering. Special Topic: Impact of Networking.

[12]  Patrice Godefroid,et al.  Model checking for programming languages using VeriSoft , 1997, POPL '97.

[13]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[14]  Klaus Havelund,et al.  Model checking programs , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[15]  Jeffrey M. Thompson,et al.  Structuring Formal Control Systems Specifications for Reuse: Surviving Hardware Changes , 2000 .

[16]  Sanjai Rayadurgam,et al.  Coverage based test-case generation using model checkers , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[17]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[18]  Paul Ammann,et al.  A SPECIFICATION-BASED COVERAGE METRIC TO EVALUATE TEST SETS , 2001 .

[19]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..