Comparison of fault classes in specification-based testing

Abstract Our results extending Kuhn's fault class hierarchy provide a justification for the focus of fault-based testing strategies on detecting particular faults and ignoring others. We develop a novel analytical technique which allows us to elegantly prove that the hierarchy applies to arbitrary expressions, not just those in disjunctive normal form. We also use the technique to extend the hierarchy to a wider range of fault classes. To demonstrate broad applicability, we compare faults in practical situations and analyze previous results. In particular, using our technique, we show that the basic meaningful impact strategy of Weyuker et al. tests for stuck-at faults, not just variable negation faults.

[1]  Yuen-Tak Yu,et al.  On the relationships of faults for Boolean specification based testing , 2001, Proceedings 2001 Australian Software Engineering Conference.

[2]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[3]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[4]  David R. Kuhn Fault Classes and Error Detection in Specification Based Testing | NIST , 1998 .

[5]  S. L. Gerhart,et al.  Toward a theory of test data selection , 1975, IEEE Transactions on Software Engineering.

[6]  B GoodenoughJohn,et al.  Toward a theory of test data selection , 1975 .

[7]  A. Jefferson Offutt,et al.  A semantic model of program faults , 1996, ISSTA '96.

[8]  Debra J. Richardson,et al.  An Analysis of Test Data Selection Criteria Using the RELAY Model of Fault Detection , 1993, IEEE Trans. Software Eng..

[9]  Steven P. Miller,et al.  Applicability of modified condition/decision coverage to software testing , 1994, Softw. Eng. J..

[10]  Nancy G. Leveson,et al.  Requirements Specification for Process-Control Systems , 1994, IEEE Trans. Software Eng..

[11]  Tarak Shantilal Goradia,et al.  Dynamic Impact Analysis: Analyzing Error Propagation in Program Executions , 1993 .

[12]  Tsong Yueh Chen,et al.  Test case selection strategies based on Boolean specifications ‡ , 2001, Softw. Test. Verification Reliab..

[13]  Elaine J. Weyuker,et al.  Automatically Generating Test Data from a Boolean Specification , 1994, IEEE Trans. Software Eng..

[14]  Timothy A. Budd,et al.  Program Testing by Specification Mutation , 1985, Comput. Lang..

[15]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[16]  D. Richard Kuhn Fault classes and error detection capability of specification-based testing , 1999, TSEM.

[17]  Larry Joe Morell A theory of error-based testing , 1983 .

[18]  Kuo-Chung Tai,et al.  Theory of Fault-Based Predicate Testing for Computer Programs , 1996, IEEE Trans. Software Eng..

[19]  Kenneth A. Foster Error Sensitive Test Cases Analysis (ESTCA) , 1980, IEEE Transactions on Software Engineering.

[20]  A Jeeerson Ooutt,et al.  Subsumption of Condition Coverage Techniques by Mutation Testing , 1996 .

[21]  Vadim Okun,et al.  Mutation operators for specifications , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[22]  Steven J. Zeil Perturbation Techniques for Detecting Domain Errors , 1989, IEEE Trans. Software Eng..

[23]  Tatsuhiro Tsuchiya,et al.  On fault classes and error detection capability of specification-based testing , 2002, TSEM.

[24]  D. Richard Kuhn A Technique for Analyzing the Effects of Changes in Formal Specifications , 1992, Comput. J..

[25]  I. S. Reed Boolean Difference Calculus and Fault Finding , 1973 .

[26]  Shaoying Liu,et al.  Criteria for generating specification-based tests , 1999, Proceedings Fifth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'99) (Cat. No.PR00434).

[27]  Lori A. Clarke,et al.  A Formal Evaluation of Data Flow Path Selection Criteria , 1989, IEEE Trans. Software Eng..