FTScMES: A New Mutation Execution Strategy Based on Failed Tests' Mutation Score for Fault Localization

Fault localization has been one of the most expensive activity in the whole debugging process. The spectrum-based fault localization (SBFL) is the most studied and evaluated technique. Other approach is the mutation-based fault localization technique (MBFL) that needs to execute the test suite on a large amount of mutants increasing its cost. Efforts from research community have been performed in order to achieve solutions reducing such cost and keeping a minimum quality. Current mutation execution strategies are evaluated considering artificial faults. However, recent researches show that some MBFL techniques exhibit low efficacy fault localization when evaluated on real faults. In this paper, we propose a new mutation execution strategy based on failed tests’ mutation score, called (FTScMES), aiming to increase the efficacy on fault localization reducing the cost of mutants execution. FTScMES uses only the set of failed test cases to execute mutants and bases on mutation score concept to compute the suspiciousness statements. The experiments were conducted considering 221 real faults, comparing the efficacy of localization of FTScMES against 5 baselines from the literature. We found that FTScMES outperforms the baselines reducing the cost in 90% on average with a high efficacy of ranking defective code.

[1]  Rui Abreu,et al.  GZoltar: an eclipse plug-in for testing and debugging , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[2]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[3]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[4]  Yves Le Traon,et al.  Effective fault localization via mutation analysis: a selective mutation approach , 2014, SAC.

[5]  Mark Harman,et al.  Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation , 2010, J. Syst. Softw..

[6]  Zheng Li,et al.  Faster mutation-based fault localization with a novel mutation execution strategy , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[7]  Yong Liu,et al.  Statement-Oriented Mutant Reduction Strategy for Mutation Based Fault Localization , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[8]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[9]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[10]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[11]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[12]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[13]  Michael D. Ernst,et al.  Evaluating and Improving Fault Localization , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[14]  W. Eric Wong,et al.  The DStar Method for Effective Software Fault Localization , 2014, IEEE Transactions on Reliability.

[15]  Shin Yoo,et al.  Ask the Mutants: Mutating Faulty Programs for Fault Localization , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[16]  René Just,et al.  The major mutation framework: efficient and scalable mutation analysis for Java , 2014, ISSTA 2014.

[17]  Shin Yoo,et al.  FLUCCS: using code and change metrics to improve fault localization , 2017, ISSTA.

[18]  Lei Zhao,et al.  A Crosstab-based Statistical Method for Effective Fault Localization , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.