An empirical evaluation of test case filtering techniques based on exercising complex information flows

Some software defects trigger failures only when certain complex information flows occur within the software. Profiling and analyzing such flows therefore provides a potentially important basis for filtering test cases. We report the results of an empirical evaluation of several test case filtering techniques that are based on exercising complex information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising basic blocks, branches, function calls, and def-use pairs, with respect to their effectiveness for revealing defects.

[1]  David Leon,et al.  Detecting and debugging insecure information flows , 2004, 15th International Symposium on Software Reliability Engineering.

[2]  David Leon,et al.  A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[3]  Bin Wang,et al.  Automated support for classifying software failure reports , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[4]  A. Porter,et al.  A history-based test prioritization technique for regression testing in resource constrained environments , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[5]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[6]  David Leon,et al.  Pursuing failure: the distribution of program failures in a profile space , 2001, ESEC/FSE-9.

[7]  David Leon,et al.  Finding failures by cluster analysis of execution profiles , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[8]  Gregg Rothermel,et al.  An empirical study of regression test selection techniques , 2001, ACM Trans. Softw. Eng. Methodol..

[9]  Gregg Rothermel,et al.  An empirical investigation of the relationship between spectra differences and regression faults , 2000, Softw. Test. Verification Reliab..

[10]  Phyllis G. Frankl,et al.  Comparison of delivered reliability of branch, data flow and operational testing: A case study , 2000, ISSTA '00.

[11]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[12]  Lee J. White,et al.  Multivariate visualization in observation-based testing , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[13]  Phyllis G. Frankl,et al.  Further empirical studies of test effectiveness , 1998, SIGSOFT '98/FSE-6.

[14]  Joseph Robert Horgan,et al.  Test set size minimization and fault detection effectiveness: a case study in a space application , 1997, Proceedings Twenty-First Annual International Computer Software and Applications Conference (COMPSAC'97).

[15]  Dorit S. Hochba,et al.  Approximation Algorithms for NP-Hard Problems , 1997, SIGA.

[16]  Rajiv Gupta,et al.  Program Slicing‐Based Regression Testing Techniques , 1996, Softw. Test. Verification Reliab..

[17]  Joseph Robert Horgan,et al.  Effect of Test Set Minimization on Fault Detection Effectiveness , 1995, 1995 17th International Conference on Software Engineering.

[18]  Bogdan Korel,et al.  Forward computation of dynamic program slices , 1994, ISSTA '94.

[19]  Gregg Rothermel,et al.  Selecting tests and identifying test coverage requirements for modified software , 1994, ISSTA '94.

[20]  Joseph Robert Horgan,et al.  Incremental regression testing , 1993, 1993 Conference on Software Maintenance.

[21]  Gregg Rothermel,et al.  A safe, efficient algorithm for regression test selection , 1993, 1993 Conference on Software Maintenance.

[22]  Lori A. Clarke,et al.  An information flow model of fault detection , 1993, ISSTA '93.

[23]  Susan Horwitz,et al.  Incremental program testing using program dependence graphs , 1993, POPL '93.

[24]  Pierre Falzon,et al.  Institut national de recherche en informatique et en automatique , 1992 .

[25]  Lori A. Clarke,et al.  A Formal Model of Program Dependences and Its Implications for Software Testing, Debugging, and Maintenance , 1990, IEEE Trans. Software Eng..

[26]  Joseph Robert Horgan,et al.  Dynamic program slicing , 1990, PLDI '90.

[27]  Lori A. Clarke,et al.  A Formal Evaluation of Data Flow Path Selection Criteria , 1989, IEEE Trans. Software Eng..

[28]  Bogdan Korel The Program Dependence Graph in Static Program Testing , 1987, Inf. Process. Lett..

[29]  Joe D. Warren,et al.  The program dependence graph and its use in optimization , 1984, TOPL.

[30]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[31]  Simeon C. Ntafos,et al.  On Required Element Testing , 1984, IEEE Transactions on Software Engineering.

[32]  Mark Weiser,et al.  Program Slicing , 1981, IEEE Transactions on Software Engineering.

[33]  Janusz W. Laski,et al.  A Data Flow Oriented Program Testing Strategy , 1983, IEEE Transactions on Software Engineering.

[34]  Peter J. Denning,et al.  Certification of programs for secure information flow , 1977, CACM.