An In-Depth Study of the Efficiency of Risk Evaluation Formulas for Multi-Fault Localization

A large amount of risk evaluation formulas have been proposed for spectrum-based fault localization (SBFL) in prior studies. A recent study by Xie et al. developed an innovative framework to theoretically analyze the effectiveness of those risk evaluation formulas in SBFL. Xie et al.'s study was based on the assumption that program has only one fault. In other words, they investigated SBFL in the context of single-fault SBFL. However, in practice, programs might have more than one faults. In this paper, we first propose a novel theoretical analysis framework for the risk evaluation formulas in the context of multi-faults SBFL. Our framework is based on a new effort cost evaluation L-Score. By applying our framework, we then conduct case studies to investigate the efficiency of existing formulas in the context of multi-faults SBFL.

[1]  Yuen-Tak Yu,et al.  Extending the Theoretical Fault Localization Effectiveness Hierarchy with Empirical Results at Different Code Abstraction Levels , 2014, 2014 IEEE 38th Annual Computer Software and Applications Conference.

[2]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[3]  Jason Lee,et al.  Improving spectral‐based fault localization using static analysis , 2017, Softw. Pract. Exp..

[4]  Lee Naish,et al.  Study of the relationship of bug consistency with respect to performance of spectra metrics , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[5]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[6]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[7]  Michael D. Ernst,et al.  Evaluating and Improving Fault Localization , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[8]  Byoungju Choi,et al.  A family of code coverage-based heuristics for effective fault localization , 2010, J. Syst. Softw..

[9]  Wes Masri,et al.  Prevalence of coincidental correctness and mitigation of its impact on fault localization , 2014, TSEM.

[10]  Baowen Xu,et al.  A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization , 2013, TSEM.

[11]  Mark Harman,et al.  Provably Optimal and Human-Competitive Results in SBSE for Spectrum Based Fault Localisation , 2013, SSBSE.

[12]  Lu Zhang,et al.  On similarity-awareness in testing-based fault localization , 2008, Automated Software Engineering.

[13]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[14]  Shin Yoo,et al.  Evolving Human Competitive Spectra-Based Fault Localisation Techniques , 2012, SSBSE.

[15]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.