Axiomatic Assessment of Logic Coverage Software Testing Criteria

Since formal specifications precisely describe the software requirements in a form that can be automatically manipulated, they can be used as the base for automatic test generation and software verification. Logic coverage criteria are the common criteria used in specification-based testing. The main problem of applying these criteria that test engineers face is how to appropriately select each criterion. Thus, the comparison and analysis of these criteria will give a guide to applying each criterion. Axiomatic assessment of test adequacy criteria is an approach to comparing test criteria. This approach defines the intuitive requirements of ideal test adequacy criteria as some axioms, then compares the test adequacy criteria by checking if they satisfy these axioms. This paper proposes some positive properties as the intuitive requirements of ideal logic coverage criteria, and gives a generating algorithm that is used to determine whether a logic coverage criterion is complete. These properties are formally defined as an axioms system. With these formal definitions, the relations among the logic coverage criteria are described as some theorems. Finally, the common logic coverage criteria are assessed against the axioms system. From the assessing result, testers can get some conclusions that help them apply these criteria in practice.

[1]  Stuart H. Zweben,et al.  Clarifying Some Fundamental Concepts in Software Testing , 1993, IEEE Trans. Software Eng..

[2]  Elaine J. Weyuker,et al.  Axiomatizing software test data adequacy , 1986, IEEE Transactions on Software Engineering.

[3]  Vadim Okun,et al.  Mutation operators for specifications , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[4]  Stuart H. Zweben,et al.  Analysis and Refinement of Software Test Data Adequacy Properties , 1991, IEEE Trans. Software Eng..

[5]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[6]  S. L. Gerhart,et al.  Toward a theory of test data selection , 1975, IEEE Transactions on Software Engineering.

[7]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[8]  Paul Ammann,et al.  Using formal methods to derive test frames in category-partition testing , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[9]  A. Jefferson Offutt,et al.  Coverage criteria for logical expressions , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[10]  John S. Gourlay A Mathematical Framework for the Investigation of Testing , 1983, IEEE Transactions on Software Engineering.

[11]  Hong Zhu Axiomatic assessment of control flow-based software test adequacy criteria , 1995, Softw. Eng. J..

[12]  Robert M. Hierons Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[13]  Antoni Diller,et al.  Z - an introduction to formal methods , 1990 .