An Evaluation of the Minimal-MUMCUT Logic Criterion and Prime Path Coverage

This paper presents comparisons of the MinimalMUMCUT logic criterion and prime path coverage. A theoretical comparison of the two criteria is performed in terms of (1) how well tests satisfying one criterion satisfy the other and (2) fault detection. We then compare the criteria experimentally. For 22 programs, we develop tests to satisfy Minimal-MUMCUT and prime path coverage. We use these tests in two separate experiments. First we measure the effectiveness of the tests developed for one criterion in terms of the other. Next we investigate the ability of the test sets to find actual faults. Faults are seeded via a mutation tool and then supplemented with mutants created by DNF logic mutation operators. We then measure the number of non-equivalent mutants killed by each test set. Results indicate that while prime path-adequate test sets are closer to satisfying Minimal-MUMCUT than vice versa, the criteria had similar fault detection and MinimalMUMCUT required fewer tests.

[1]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997, Softw. Test. Verification Reliab..

[2]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[3]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[4]  W. E. Riggs.  An Experimental Evaluation , 1983 .

[5]  A. Jefferson Offutt,et al.  An Experimental Evaluation of Data Flow and Mutation Testing , 1996 .

[6]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[7]  R. Lipton,et al.  Mutation analysis , 1998 .

[8]  W. Eric Wong,et al.  An empirical comparison of data flow and mutation‐based test adequacy criteria , 1994, Softw. Test. Verification Reliab..

[9]  Mary Jean Harrold,et al.  Fault modeling using the program dependence graph , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[10]  MaYu-Seung,et al.  MuJava: an automated class mutation system , 2005 .

[11]  Elaine J. Weyuker,et al.  Comparison of program testing strategies , 1991, TAV4.

[12]  Paul Ammann,et al.  Reconciling perspectives of software logic testing , 2008 .

[13]  Paul Ammann,et al.  Using Logic Criterion Feasibility to Reduce Test Set Size While Guaranteeing Fault Detection , 2009, 2009 International Conference on Software Testing Verification and Validation.

[14]  A Jeeerson Ooutt,et al.  Subsumption of Condition Coverage Techniques by Mutation Testing , 1996 .

[15]  Paul Ammann,et al.  Reducing logic test set size while preserving fault detection , 2011, Softw. Test. Verification Reliab..