FTMES: A Failed-Test-Oriented Mutant Execution Strategy for Mutation-Based Fault Localization

Fault localization has been one of the most manual and costly software debugging activities. The spectrum-based fault localization is the most studied and evaluated fault localization approach. Mutation-based fault localization is a promising approach but with a high computational cost. We propose a novel mutation execution strategy named Failed-Test Oriented Mutant Execution Strategy (FTMES) for improving the efficacy of fault localization techniques and also reduce the computational cost of these techniques. Our proposed approach and eight other baselines were evaluated against 221 real faults. The results show that FTMES outperformed others with respect to efficiency (computational cost) while maintaining similar accuracy.

[1]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[2]  W. Eric Wong,et al.  The DStar Method for Effective Software Fault Localization , 2014, IEEE Transactions on Reliability.

[3]  Shin Yoo,et al.  Ask the Mutants: Mutating Faulty Programs for Fault Localization , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[4]  René Just,et al.  Higher accuracy and lower run time: efficient mutation analysis using non‐redundant mutation operators , 2015, Softw. Test. Verification Reliab..

[5]  Rui Abreu,et al.  Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators , 2013, ISSTA.

[6]  Celso G. Camilo-Junior,et al.  Evolutionary Composition of Customized Fault Localization Heuristics , 2018, ESANN.

[7]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[8]  Michael D. Ernst,et al.  Evaluating and Improving Fault Localization , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[9]  Yves Le Traon,et al.  Metallaxis‐FL: mutation‐based fault localization , 2015, Softw. Test. Verification Reliab..

[10]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[11]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[12]  Lei Zhao,et al.  A Crosstab-based Statistical Method for Effective Fault Localization , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[13]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[14]  Martin Monperrus,et al.  Test case purification for improving fault localization , 2014, SIGSOFT FSE.

[15]  René Just,et al.  The major mutation framework: efficient and scalable mutation analysis for Java , 2014, ISSTA 2014.

[16]  Shin Yoo,et al.  FLUCCS: using code and change metrics to improve fault localization , 2017, ISSTA.

[17]  Claire Le Goues,et al.  Improved representation and genetic operators for linear genetic programming for automated program repair , 2017, Empirical Software Engineering.

[18]  Celso G. Camilo-Junior,et al.  Mutation-Based Evolutionary Fault Localisation , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[19]  Yong Liu,et al.  An optimal mutation execution strategy for cost reduction of mutation-based fault localization , 2018, Inf. Sci..

[20]  Xiang Ji,et al.  A Test Suite Reduction Approach to Improving the Effectiveness of Fault Localization , 2017, 2017 International Conference on Software Analysis, Testing and Evolution (SATE).

[21]  Zheng Li,et al.  Faster mutation-based fault localization with a novel mutation execution strategy , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[22]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[23]  W. Eric Wong.  Mutation Testing for the New Century , 2001 .

[24]  Yong Liu,et al.  Statement-Oriented Mutant Reduction Strategy for Mutation Based Fault Localization , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[25]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[26]  Wes Masri,et al.  Fault localization based on information flow coverage , 2010, Softw. Test. Verification Reliab..

[27]  W. Eric Wong,et al.  Software Fault Localization Using DStar (D*) , 2012, 2012 IEEE Sixth International Conference on Software Security and Reliability.

[28]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[29]  Andreas Zeller,et al.  Lightweight bug localization with AMPLE , 2005, AADEBUG'05.

[30]  Rui Abreu,et al.  GZoltar: an eclipse plug-in for testing and debugging , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[31]  Yves Le Traon,et al.  Effective fault localization via mutation analysis: a selective mutation approach , 2014, SAC.

[32]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[33]  Auri Marcelo Rizzo Vincenzi,et al.  A coevolutionary algorithm to automatic test case selection and mutant in Mutation Testing , 2013, 2013 IEEE Congress on Evolutionary Computation.

[34]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.