An evaluation of Boolean expression testing techniques

Increase in the size and complexity of the software developed has made software testing a challenging exercise. A number of testing techniques are available but they differ in terms of statement coverage, condition coverage and particularly in fault detection capabilities. The size of the test suite also differs from one technique to other. Fault that has propagated into the system inadvertently, especially into the branch statements, have severe effects as they affect the logic of the program. In this paper, an experimental evaluation of the popular branch-testing techniques (Elmendorf's method, Boolean Operator (BOR), Modified Condition/Decision Coverage (MCDC), and Reinforced Criteria/Decision Coverage (RCDC)) is presented. These techniques are evaluated on the basis of types of faults they identify, size of the test suite and their effectiveness in fault detection. For experiments, various branch statements used and referred in literature are selected. Test cases and mutants were prepared for these branch statements. Mutants were prepared by seeding single operator and operand faults into the statements. The results indicate that for a subset of fault types BOR is effective. A variant of MCDC and RCDC demonstrate better performance on the full class of faults and are only slightly worse than Elmendorf's (CEG) method test suite.

[1]  A. L. White Comments on modified condition/decision coverage for software testing [of flight control software] , 2001, 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542).

[2]  John Joseph Chilenski,et al.  An Investigation of Three Forms of the Modified Condition Decision Coverage (MCDC) Criterion , 2001 .

[3]  Jonathan P. Bowen,et al.  Experimental evaluation of the variation in effectiveness for DC, FPC and MC/DC test criteria , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[4]  A. Jefferson Offutt,et al.  An Approach to Fault Modeling and Fault Seeding Using the Program Dependence Graph , 1997, J. Syst. Softw..

[5]  Tatsuhiro Tsuchiya,et al.  On fault classes and error detection capability of specification-based testing , 2002, TSEM.

[6]  Steven P. Miller,et al.  Applicability of modified condition/decision coverage to software testing , 1994, Softw. Eng. J..

[7]  Kalpesh Kapoor Stability of test criteria and fault hierarchies in software testing , 2004 .

[8]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[9]  A. Jefferson Offutt,et al.  Investigations of the software testing coupling effect , 1992, TSEM.

[10]  Mladen A. Vouk,et al.  Automatic test-generation for predicates [software testing] , 1996, IEEE Trans. Reliab..

[11]  Elaine J. Weyuker,et al.  Automatically Generating Test Data from a Boolean Specification , 1994, IEEE Trans. Software Eng..

[12]  Mladen A. Vouk,et al.  Evaluation of a Predicate-Based Software Testing Strategy , 1994, IBM Syst. J..

[13]  Jonathan P. Bowen,et al.  Experimental evaluation of the tolerance for control‐flow test criteria , 2004, Softw. Test. Verification Reliab..

[14]  Kuo-Chung Tai,et al.  Theory of Fault-Based Predicate Testing for Computer Programs , 1996, IEEE Trans. Software Eng..

[15]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[16]  Mladen A. Vouk,et al.  Empirical studies of predicate-based software testing , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[17]  Jonathan P. Bowen,et al.  Tolerance of control-flow testing criteria , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[18]  Mladen A. Vouk,et al.  Automatic test generation for predicates , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[19]  Mary Jean Harrold,et al.  Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage , 2003, IEEE Trans. Software Eng..

[20]  Kuo-Chung Tai,et al.  Test generation for Boolean expressions , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[21]  Jonathan P. Bowen,et al.  Reinforced Condition/Decision Coverage (RC/DC): A New Criterion for Software Testing , 2002, ZB.

[22]  Elaine J. Weyuker,et al.  A Formal Analysis of the Fault-Detecting Ability of Testing Methods , 1993, IEEE Trans. Software Eng..

[23]  Sigrid Eldh Software Testing Techniques , 2007 .