Comparing Multi-Point Stride Coverage and dataflow coverage

We introduce a family of coverage criteria, called Multi-Point Stride Coverage (MPSC). MPSC generalizes branch coverage to coverage of tuples of branches taken from the execution sequence of a program. We investigate its potential as a replacement for dataflow coverage, such as def-use coverage. We find that programs can be instrumented for MPSC easily, that the instrumentation usually incurs less overhead than that for def-use coverage, and that MPSC is comparable in usefulness to def-use in predicting test suite effectiveness. We also find that the space required to collect MPSC can be predicted from the number of branches in the program.

[1]  Akbar Siami Namin,et al.  The influence of size and coverage on test suite effectiveness , 2009, ISSTA.

[2]  W. Holtzman.  Fundamental statistics in psychology and education. , 1951 .

[3]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[4]  Ajitha Rajan,et al.  The effect of program and model structure on mc/dc test adequacy coverage , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[5]  Gregg Rothermel,et al.  On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques , 2006, IEEE Transactions on Software Engineering.

[6]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[7]  Martin R. Woodward,et al.  On the relationship between two control-flow coverage criteria: all JJ-paths and MCDC , 2006, Inf. Softw. Technol..

[8]  J.-C. Rault,et al.  A software reliability assessment based on a structural and behavioral analysis of programs , 1976, ICSE '76.

[9]  Joseph Robert Horgan,et al.  Data flow coverage and the C language , 1991, TAV4.

[10]  Gregg Rothermel,et al.  An empirical study of the effects of minimization on the fault detection capabilities of test suites , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[11]  Qian Yang,et al.  A survey of coverage based testing tools , 2006, AST '06.

[12]  Elaine J. Weyuker,et al.  Provable Improvements on Branch Testing , 1993, IEEE Trans. Software Eng..

[13]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[14]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[15]  Stefan Berner,et al.  Enhancing Software Testing by Judicious Use of Code Coverage Information , 2007, 29th International Conference on Software Engineering (ICSE'07).

[16]  Simeon C. Ntafos,et al.  On Required Element Testing , 1984, IEEE Transactions on Software Engineering.

[17]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[18]  Martin R. Woodward,et al.  Experience with Path Analysis and Testing of Programs , 1980, IEEE Transactions on Software Engineering.

[19]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[20]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[21]  Raúl A. Santelices,et al.  Efficiently monitoring data-flow test coverage , 2007, ASE.

[22]  Ryan Womack,et al.  Introduction to R , 2010, IASSIST.

[23]  Bertrand Meyer,et al.  ARTOO , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[24]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[25]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[26]  Tim Menzies,et al.  Nighthawk: a two-level genetic-random unit test data generator , 2007, ASE.

[27]  J. P. Guilford,et al.  Fundamental statistics in psychology and education , 1943 .

[28]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[29]  Thomas Ball,et al.  A Theory of Predicate-Complete Test Coverage and Generation , 2004, FMCO.

[30]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[31]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[32]  J. R. Horgan,et al.  A data flow coverage testing tool for C , 1992, [1992] Proceedings of the Second Symposium on Assessment of Quality Software Development Tools.

[33]  A. Jefferson Offutt,et al.  Mutation 2000: uniting the orthogonal , 2001 .