Controlling test case explosion in test generation from B formal models

BZ‐TESTING‐TOOLS (BZ‐TT) is a tool set for automated test case generation from B and Z specifications. BZ‐TT uses boundary and cause–effect testing on the basis of the formal model. It has been used and validated on several industrial applications in the domain of critical software, particularly smart card and transport systems. This paper presents the test coverage criteria supported by BZ‐TT. On the one hand, these correspond to various classical structural coverage criteria, but specialized to the case of B abstract machines. The paper gives algorithms for these in Prolog. On the other hand, BZ‐TT introduces new coverage criteria for complex data structures, based on boundary analysis: this paper defines weak and strong state‐boundary coverage, input‐boundary coverage and output‐boundary coverage. Finally, the paper describes how BZ‐TT presents a unified view of these criteria to the validation engineer, and allows him or her to control the test case explosion on a coarse basis (choosing from a range of coverage criteria) as well as a fine basis (selecting options for each state or input variable). Copyright © 2004 John Wiley & Sons, Ltd.

[1]  Paul Strooper,et al.  From Object‐Z specifications to ClassBench test suites , 2000 .

[2]  Boundary values and automated component testing , 1999 .

[3]  David A. Carrington,et al.  Test templates: a specification-based testing framework , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[4]  Daniel Le Métayer,et al.  CASTING: a formally based software test generation method , 1997, First IEEE International Conference on Formal Engineering Methods.

[5]  Margus Veanes,et al.  Generating finite state machines from abstract state machines , 2002, ISSTA '02.

[6]  Jonathan P. Bowen,et al.  Formalization of software testing criteria using the Z notation , 2001, 25th Annual International Computer Software and Applications Conference. COMPSAC 2001.

[7]  Lee J. White,et al.  A Domain Strategy for Computer Program Testing , 1980, IEEE Transactions on Software Engineering.

[8]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[9]  Bogdan Korel,et al.  Black-box test reduction using input-output analysis , 2000, ISSTA '00.

[10]  Alan Hartman,et al.  Using a model-based test generator to test for standard conformance , 2002, IBM Syst. J..

[11]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[12]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[13]  Bruno Legeard,et al.  Automated Boundary Testing from Z and B , 2002, FME.

[14]  Mark Harman,et al.  Conditioned slicing supports partition testing , 2002, Softw. Test. Verification Reliab..

[15]  Hélène Waeselynck,et al.  Test Criteria Definition for B Models , 1999, World Congress on Formal Methods.

[16]  Fabrice Bouquet,et al.  CLPS-B - A Constraint Solver for B , 2002, TACAS.

[17]  Fabrice Bouquet,et al.  BZ-testing-tools: A tool-set for test generation from Z and B using constraint logic programming , 2002 .

[18]  Bogdan Korel,et al.  Dependence analysis in reduction of requirement based test suites , 2002, ISSTA '02.

[19]  Jean-Raymond Abrial,et al.  The B-book - assigning programs to meanings , 1996 .

[20]  Hoyt Lougee,et al.  SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION , 2001 .

[21]  Bruno Legeard,et al.  Generation of test sequences from formal specifications: GSM 11‐11 standard case study , 2004, Softw. Pract. Exp..

[22]  Fabrice Bouquet,et al.  Reification of Executable Test Scripts in Formal Specicifation-Based Test Generation: The Java Card Transaction Mechanism Case Study , 2003, FME.

[23]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[24]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[25]  Robert M. Hierons.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[26]  Bruno Legeard,et al.  Generation of functional test sequences from B formal specifications presentation and industrial case-study , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[27]  Bruno Legeard,et al.  Preamble computation in automated test case generation using constraint logic programming , 2004, Softw. Test. Verification Reliab..

[28]  Alan Hartman,et al.  Projected state machine coverage for software testing , 2002, ISSTA '02.

[29]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.