Automated Search for Good Coverage Criteria: Moving from Code Coverage to Fault Coverage through Search-Based Software Engineering

We propose to use Search-Based Software Engineering to automatically evolve coverage criteria that are well correlated with fault revelation, through the use of existing fault databases. We explain how problems of bloat and overfitting can be ameliorated in our approach, and show how this new method will yield insight into faults --- as well as better guidance for Search-Based Software Testing.

[1]  Phil McMinn,et al.  The Effectiveness of Test Coverage Criteria for Relational Database Schema Integrity Constraints , 2015, ACM Trans. Softw. Eng. Methodol..

[2]  Gordon Fraser,et al.  Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[3]  Gordon Fraser,et al.  Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study , 2015, ACM Trans. Softw. Eng. Methodol..

[4]  Gregory Gay,et al.  The Risks of Coverage-Directed Test Case Generation , 2015, IEEE Transactions on Software Engineering.

[5]  Mark Harman,et al.  Ieee Transactions on Evolutionary Computation 1 , 2022 .

[6]  Abhik Roychoudhury,et al.  CoREBench: studying complexity of regression errors , 2014, ISSTA 2014.

[7]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[8]  Reid Holmes,et al.  Coverage is not strongly correlated with test suite effectiveness , 2014, ICSE.

[9]  Gordon Fraser,et al.  Does automated white-box test generation really help software testers? , 2013, ISSTA.

[10]  Alessandro Orso,et al.  Search-Based Propagation of Regression Faults in Automated Regression Testing , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[11]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[12]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[13]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[14]  Luciano Lavagno,et al.  Electronic Design Automation for Integrated Circuits Handbook , 2006 .

[15]  Mark Harman,et al.  Search-based amorphous slicing , 2005, 12th Working Conference on Reverse Engineering (WCRE'05).

[16]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[17]  Jane Huffman Hayes.  Building a requirement fault taxonomy: experiences from a NASA verification and validation research project , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[18]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[19]  Tatsuhiro Tsuchiya,et al.  On fault classes and error detection capability of specification-based testing , 2002, TSEM.

[20]  L. Altenberg.  EMERGENT PHENOMENA IN GENETIC PROGRAMMING , 1994 .

[21]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..